Dion Almaer

Software, Development, Products

Stitching with the new Jules API

The Jules team finished off another ship week with the release of an initial version of their API, allowing you to integrate with your friendly asynchronous squid engineer.

I have been interested in having a native app that let’s me interact with Jules, and get notifications pinged to me when activities are ready for the next step. This way I can nudge the system along before the next thing.

So, I found myself in front of some English Premier League games on a Sunday morning, laptop on my… well lap.

By the time the last ball was kicked in the Brentford vs. Man City game, I had a working application that was designed by Stitch, and coded up with Jules… a favorite pair of mine these days 🙂

Starting with the Spec

As per usual these days, I started by defining a simple spec that ​captures what I am looking to build: it’s design and implementation. Sometimes I work with an AI to go deep on the definition up front, and other times I stay shallower and flush out the spec as I add more features.

This is also where I capture useful context such as the documentation for the alpha Jules API, so it is always available to an agent.

Designed By Stitch

Next, I feed the spec, which contains the screens that I am looking to design, into Stitch. Given that Jules has a strong design aesthetic, I could take a screenshot of the home page, which includes all of the whimsy of the squid and their adventures, and pass it into Stitch as the design inspiration to go along with the spec.

That is all I needed to get what I wanted this time, in comparison to when I am in a new area exploring a bunch of design styles.

I tend to follow a layout pattern for screens when there isn’t a complex flow.

The first screen on the left is the screen I ended up choosing, and the others to the right are copies where I have asked for specific edits, or where I have asked for a series of variants. It’s a rare project when I don’t ask for variants as this is the fun of Stitch! It’s cheap to explore!

And here’s the project to explore yourself!

Coded by Jules

Finally, it’s time to build. I setup a repo that contains my spec as a README.md, and the screens that I want to build which have been downloaded from Stitch.

This time, I decided to code a SwiftUI based native iPhone app. Hmm, the Stitch screens are paired images as well as HTML/CSS code. Fortunately, LLMs are REALLY good at translating. They can listen in English and speak back in Spanish. And they can read HTML and come back with Swift. It’s impressive.

Now, I have the admit that the early alpha API doesn’t have allllll of the documentation available yet, so I first asked for an API client, and then had it run through and output sample payloads which I saved away to look through, and for the AI to have in it’s back pocket to flush out the full client with the given info.

Now I had designs, API samples, docs, and a spec. It was time to get Jules coding… and while waiting for some of the initial work, I noodled away and added issues to tag Jules into later.

Now, I admit that it’s a bit more frustrating working with an iPhone app compared to the Web… I wasn’t able to puppeteer, nor get preview links to test with in a wonderful happy cloud. Instead I was running locally and pulling things down (and back with changes).

But a short time later, I was enjoying a new shiny squid app on my phone. I love this new world of personal software where you can go from an idea to it running in your hand, in the exact way you want it.

Here are some of the screens in all their glory!

More adventures with my friend are just a tap away!

/fin

Pools of Extraction: How I Hack on Software Projects with LLMs

I’ve started to notice some patterns in how I work on software projects.

Specs as mini-pools

I often sit down and just write a spec. Even if I never build it, the spec itself is a kind of temporary pool: a place where scattered ideas turn into something concrete enough to extract from later.

When I say “write a spec”, I actually start with a “brief” and the spec flushes out over time. AI helps to build out the spec through back and forth, and by reverse-extracting changes to a system. I’m excited for more tools that will help with extraction and syncing. If we can get there, specs can become the source of truth.

It’s important to do so. I was talking to a friend recently who works on spec driven systems and we kept discussing the analogy of building a house.

Imagine meeting with your architect and having a conversation about what you want to build. It starts high level, and the architect comes back with options which you iterate on together. The architect flushes out actual designs, and you can communicate around the nice 3D models vs. the blue prints.

Now imagine meeting with the GC and you just start riffing on what you want to built. Obviously crazy… things get written down and everyone aligns around the same shared sources.

We also have inspectors to verify that that build matches the design and follows all of the building codes. Yes, we have building codes that are shared sources of truth and constraints.

Right now we are still relying on a lot of “talk to the builder and they go off and try to build what you just said” with chat based agents. Over time we will add more shared sources, will have better verification, and this will enable us to scale what we build.

Faster cheaper smaller models

We often think about how specifying things well leads to higher quality output from models. This is true. What is often under appreciated is that it also means that you can use a variety of models to do the job.

Asking a SOTA galaxy brain that has knowledge of Genghis Khan, every language, and on and on can often be overkill. You can do hundreds or even thousands of “flash” calls for the price of one larger call. Thus, having the right knowledge available can allow a small model to give you what you need very quickly and cheaply… especially if a fix loop is applied.

Throwaway projects

I spin up tiny projects all the time, just to try something out. Most of them don’t live very long, but that’s the point—they’re cheap experiments that give me a quick pool of learning before I move on.

Shopify’s Tobi published a repo that helps make this easy:

Repo extractions

Sometimes I yank code out of a larger repo, or glue pieces together from a few repos, and build something smaller and more focused. By narrowing the scope, I create a pool that’s easier to wade into.

Because LLMs are so great at taking different information and translating it for your needs, this can happen in interesting ways. Where I would look for more abstractions and shared libraries in the past, it’s more murky now when I can easily create exactly what I need.

It turns out, all of these are me making temporary pools of extraction. And once I noticed that pattern in my own work, I started seeing it everywhere—especially in how LLMs and even our own brains operate.

Pools of Extraction in LLMs

LLMs have two big pools to fish from:

  • The latent pool: everything baked into their weights from training. Big, deep, and fuzzy.
  • The contextual pool: whatever you feed them right now—repos, docs, examples. Sharp, specific, but fleeting.

If you ask a model a question without any context, it dives into the murky depths of its latent pool and hopes to surface something useful. Hand it your relevant code repo, and suddenly it’s swimming in a nice, clear backyard pool with everything it needs right there.

That repo context might not live beyond the session, but while it’s there, it massively increases the odds of good output.

Pools of Extraction in Humans

Humans work in a parallel but messier way. If someone asks me about a library I used years ago, I might dredge up the answer from long-term memory. But if I’ve got my old notes or an example in front of me? Way easier.

Psychologists call these retrieval cues—they reactivate the right pathways so memory reconstructs itself. And unlike LLMs, every time I do recall something, I actually change the memory. My brain’s wiring shifts a little. With LLMs, nothing changes in the weights—context pools evaporate when the session closes.

Conclusion: Build Better Pools

Specs, toy projects, repo extractions—these are all ways I give myself better pools to draw from. LLMs need the same treatment: without context they flail, with it they shine. Humans too: notes, cues, and scaffolds make the difference between blank stares and “aha!”

So now when I’m stuck, I don’t just hope the big pool will have the answer. I ask myself: what’s the smallest pool I can create right now to make extraction easier?

It’s all open book tests from here.

That’s when things click.

Stitch Design Variants: A Picture Really Is Worth a Thousand Words?

My head has been in the world of code generation for the last few years. Now that I’m also thinking about design generation, I’m seeing a lot of similarities but also some fascinating differences.

For example, it is cheap to ask AI for things. I will ask it to start from scratch and do a lot of extra work that I would never ask of my team mates.

What isn’t cheap though is the human side of reviewing all of the code that AI creates. Reading through them, parsing the differences, holding them in working memory—it’s real cognitive load. It takes time, and time is expensive.

With designs though, the calculus is a bit different. It is still cheap for AI to spin up variations, but human review? Our visual brain is ridiculously efficient. You can glance at six screens side-by-side and instantly know which one sparks “ugh” and which one sparks “oh yes!”. That “emergence” moment—that’s the magic.

Which brings me to a new Stitch feature: Design Variants. It does exactly this. When looking at one of your app UI screens, simply click on “Create variants” and you get a spread of designs.

Here you see the entropy, the unexpected angles, the one idea you wouldn’t have thought of yourself. You select, delete, iterate. Fast. Natural. Fun.

This is just the first version. We’ll be adding a lot more ways to let the AI flood the canvas with possibilities while keeping the cost of selection close to zero.

A picture is worth a thousand words. Design Variants is here to give you … ten thousand?

NOTE: The example image above is a shared public project … another feature the team shipped this week! I would love to see any of your Stitch projects!

The right thing to do, is the right thing to do.