Actually I wanted to try a bit GithubCopilot with Agents.md. Yet .. I think during the project I totally forgot to test the influence of the Agents file but tried “vibeCoding” in a reproducible way.
I had a very little project in mind that authenticates to Mastodon, fetches some data, saves into a database and displays some metrics on a web page in basic charts. Nothing overly fancy, but also some stuff that would simply take some time when coding “alone”. Like proper OAUTH flow, paging through mastodon apis, rate limiting, database writing, database setup script and cleanup. Some Javascript for the chart, etc.
But I thought it might be nice to try with GithubCopilot (GHC). But I’m also a big fan of reproducible results. So … step by step, what did I do.
Agents.md
Agents.md first. uff, do I have to search a good one? Or … maybe ask Github Copilot to scaffold an md-file and ask me questions if anything should be clarified upfront. And indeed it demanded some clarification. The Agents.md file now defines codestyle, patterns, conventions, standards, readability hints. NICE! Lots of stuff that I usually do implicitly, but the Agent probably wouldn’t. I especially love the hint “Prioritize readability over cleverness“.
It’s All About Context!
Okay, next: what exactly do I want to create (I avoid the term program because I don’t do it myself)? Let’s write a Specification.md. The Spec is .. well a specification. I tried to explain as much as I can what I want as a result. At the end, I told GHC something like “Please asses the specification. Would it be possible to generate an according project? Ask me all questions needed to clarify ambiguities and write the result back to the specification“.
Check this prompt carefully. It has two important information: First of all, I (let) do an assessment of the specification instead of just generating code. I wrote the spec as best as I could but that doesn’t mean somebody else understood what I meant. The second part is: whatever I answer, the Spec should be extended and thus the context of the project is enriched! In former projects I just clarified such questions in the chat and didn’t enrich the persistent context of the project.
Okay, so back to the project: GHC wanted ~8 (very valid) questions to be clarified. And after giving the answers it rewrote / exteded the specification accordingly. I did this once more as the context was now more informative. And again it had 5-10 valid questions to be clarified. A couple of details also came to my mind. But from this on, I didn’t change the Specification.md myself but told GHC to do that. — And I checked afterwards if the info was added to the correct sections of the file.
After this session, the Specification.md already had over 300 Lines and ~14k characters!
Environment
I already had a Docker file (and docker-compose.yaml) file from another PHP-Project that I reused for local coding. So I didn’t have to set this up.
Generate and Iterate
Now let the fun part begin! I told GHC to generate the project described in the Specification.md, watched and waited … I was reminded to the xkcd-comic “compiling”. Is this how the programming part of Software Development looks like in the future? Maybe …
The first attempt should some flaws in the OAuth flow. I copied the error message to GHC and told it to “please fix this error, and reflect the solution also in the specification”. Again: don’t just fix it but persist this information!
I went through the whole application (which I admit isn’t very large!) and did the same in a couple of other cases where the spec was either unclear or when I was able to make the spec more precise (especially as soon as I saw the gui the first time). Also the API access demanded to respect rate limits and additional exit conditions.
After a while I wasn’t sure how much my “refinement” was really reflected in the specification. Which als means: I wasn’t sure how reproducible / stable my specification was. The only way to test was … delete all generated files and tell GHC to generate the project again. Ouch, even while I type this, it still feels a bit odd. But the code was just generated from a machine in a couple of minutes. The value is in the specification.
And — if I remember correctly — it worked out well! I uploaded it to my webhost-provider, fixed some encoding things (also wrote that back into the spec.) and that’s it, it’s online.
Persist Context Whenever Possible
This was just a little project, it doesn’t have tests or so yet. But for a little “some hours” hobby project, this was enormously convenient and fast! But tests would be even more context. As they must all be green.
My main learnings were:
- Always enrich the project context! Even though it might seem tedious. The chat is transient. The specification is permanent.
- In some years we probably won’t do as much programming as we did in the past, but for SoftwareEngineering, I still see quite some need.
Leave a Reply