While working on other tasks I had a random thought: “Could the ai update my conventions using the analysis I’ve provided in the blog posts I’ve written so far?”
One Month Recap
Prompt:
read all of my blog posts in the april 2025 folder, take my analysis of what things worked, and didn’t work and make a summary for me
Details
Okay, I have read through your April 2025 “Hour a Day: AI” blog posts. Here is a summary based on your analysis (✅ for successes, ❌ for failures/issues):
Summary of AI Interactions in April 2025 Blog Posts:
What Worked Well (✅):
- Applying Explicit Conventions:
- Successfully renamed tests according to the “Should…” convention when explicitly asked (Day 7, Day 9).
- Used
var
correctly after the convention was clarified (Day 9).- Applied DRY principle by creating a constant for a filename after the convention was updated (Day 11).
- Used LINQ
OrderByDescending
for sorting (Day 17).- Scaffolding & Repetitive Tasks:
- Generated initial test stubs (though names needed refinement) (Day 7).
- Scaffolded
TearDown
methods across many Harmony mock files (Day 20).- Automated the manual refactoring of
ItemModel
constructor calls across multiple test files, saving significant time (Day 25).- Implementing Specific, Well-Defined Logic:
- Implemented specific test logic when guided (e.g., FilePathUtility tests - Day 13).
- Implemented a simple regex replacement correctly when prompted (Day 22).
- Added properties to models and updated service logic accordingly when following a plan (Day 17).
- Removed specific code blocks/functionality when asked (Day 12, Day 13).
- Added specific test cases and assertions when prompted (Day 12, Day 17, Day 22).
- Generating Ideas & Plans:
- Provided useful ideas for potential test failures (Day 10, Day 18, Day 19, Day 24).
- Generated reasonable implementation plans (Day 15, Day 17).
- Offered good options for JSON serialization with constructors (Day 25).
- Provided good options for rewording conventions (Day 15).
- Explaining Concepts & Reasoning:
- Explained why a function wasn’t pure (Day 15).
- Reasoned about test redundancy (Day 12).
- Summarized provided conventions accurately (Day 16).
- Basic Refactoring:
- Performed simple refactors like extracting methods and using early returns when asked to apply conventions (Day 11).
- Renamed methods and updated call sites (though sometimes missed spots) (Day 17).
- Handling Constraints:
- Respected the “don’t fill stubs” constraint once added (Day 9).
- Respected the “don’t change test assertions” constraint (Day 17).
- Platform-Specific Fixes (with guidance):
- Eventually fixed macOS path comparison issues in tests using a custom helper (Day 16).
What Didn’t Work Well / Required Guidance (❌):
- Applying Conventions Implicitly:
- Didn’t prioritize function purity despite the convention, requiring explicit prompts (Day 15).
- Didn’t automatically apply DRY to string literals without specific convention updates/prompts (Day 11).
- Didn’t suggest splitting classes into multiple files based on the “one class per file” convention (Day 13).
- Complex Refactoring & Design:
- Struggled with Dependency Injection concepts without explicit guidance (Day 10).
- Initial refactoring of parsing logic didn’t significantly reduce complexity (Day 15).
- Proposed overly complex solutions (e.g., test state reset manager - Day 20; markdown header removal - Day 22).
- Mocking & Testing Nuances:
- Failed to correctly mock Moq extension methods, suggesting incorrect approaches (Day 9).
- Proposed complex wrappers instead of using the library’s dedicated testing package (Spectre.Console.Testing) (Day 9).
- Made incorrect assumptions in test logic (e.g., min/max input validation - Day 10).
- Created “useless” tests that passed without proper assertion (Day 10).
- Missed asserting all relevant parts of a result (Day 12).
- Used incorrect property names in assertions (Day 9).
- Didn’t diagnose subtle test setup issues (NRE due to
[Setup]
vs[OneTimeSetup]
) (Day 19, Day 24).- Completeness & Propagation:
- Missed updating all call sites when renaming methods or changing signatures (Day 13, Day 17).
- Initial fixes sometimes incomplete (e.g., macOS path fix - Day 16).
- Over-Eagerness / Misinterpretation:
- Persistently filled in test stubs even when not asked, requiring explicit constraints (Day 7, Day 9).
- Misinterpreted prompts, implementing the wrong step of a plan (Day 17).
- Hallucinated errors or context (Day 9).
- Tooling Issues:
- The aider tool itself failed to apply generated diffs correctly, possibly due to formatting/line endings (Day 18, Day 19).
Overall Takeaways from Your Analysis:
- The AI is very effective for scaffolding, repetitive tasks, and implementing specific, well-defined logic, saving significant time.
- Explicit conventions and constraints are crucial for guiding the AI, but it may still need direct prompts to apply them, especially for design principles like purity or DRY.
- The AI struggles with complex design/refactoring, subtle testing issues (mocking, setup), and ensuring completeness across the codebase without specific guidance.
- Iterative prompting, providing context (like error messages), and breaking down tasks are necessary for complex problems.
- The ideation/planning phase with the AI is valuable, even if the initial implementation needs correction.
- Tool integration issues can sometimes impede progress even if the AI’s suggestion is correct.
- You noted the emergence of a “Convention Driven Development” workflow.
Tokens: 167k sent, 1.3k received.
Verdict: ✅ Somehow it’s still crazy to me how this is even possible. A month, of writing, analysis, and verdicts and within a minute the ai is able to summarize everything that was fed into it.
Summary into Conventions
Prompt:
given all of that and having access to my conventions, can you suggest any new conventions to add, or modify to address some of the shortcomings I found
Details
Okay, based on the shortcomings identified in your April 2025 blog posts and your existing conventions, here are some suggestions for new or modified conventions to help the AI perform better:
Proposed New Conventions:
Prioritize Library-Specific Testing Tools:
- Convention: “When testing code that interacts with external libraries (e.g., Spectre.Console, database drivers), prioritize using the library’s dedicated testing tools or packages (e.g.,
Spectre.Console.Testing
) over general mocking frameworks (like Moq) if available and appropriate for unit testing.”- Reasoning: Addresses the issue where the AI struggled with Moq for Spectre.Console’s extension methods and didn’t initially suggest the library’s testing package (Day 9). This guides it towards the intended, often simpler, testing paradigm for specific libraries.
Mandatory Call Site Updates:
- Convention: “CRITICAL: After renaming methods/properties or changing function/method signatures (parameters, return types), ALWAYS identify and update all call sites across the relevant codebase, including implementation files, tests, and any other dependent locations within the provided context.”
- Reasoning: Addresses multiple instances where the AI renamed or refactored a method/signature but missed updating all the places it was called (Day 13, Day 17), leading to compilation errors.
Assertion Completeness:
- Convention: “Test assertions MUST validate all relevant outputs and state changes of the code under test. Do not write partial assertions that only check a subset of the expected outcome unless specifically requested. Ensure assertions cover the core functionality being tested.”
- Reasoning: Addresses the creation of “useless” tests (Day 10) or tests that missed asserting key parts of the result (Day 12).
Explicit Handling of Edge Cases/Platform Differences:
- Convention: “When dealing with file paths, environment variables, or other system interactions, explicitly consider and test for platform differences (e.g., macOS vs. Windows path formats, line endings) and potential edge cases (e.g., file not found, directory permissions).”
- Reasoning: Addresses the macOS path issues that required multiple attempts to fix (Day 16) and encourages proactive thinking about cross-platform compatibility.
Simplicity First for Complex Logic:
- Convention: “When implementing or refactoring complex logic, first propose the simplest possible solution that meets requirements. Only suggest more complex patterns (e.g., custom wrappers, managers, abstract factories) if the simple approach is demonstrably insufficient or if explicitly requested.”
- Reasoning: Addresses the tendency to propose overly complex solutions (Day 20, Day 22) when a simpler approach might suffice.
Proposed Modifications to Existing Conventions:
Strengthen DRY Application (Beyond Code Structure):
- Current (Implied by
default.md
): “You are not allowed to repeat yourself…”- Modification Suggestion: “PRIORITY: Actively seek opportunities to apply DRY (Don’t Repeat Yourself) to all code elements, including logic, constants, magic strings/numbers, and configuration values. Extract duplicated values or logic into named constants, variables, or helper functions.”
- Reasoning: Addresses the failure to automatically DRY up the
".aider.chat.history.md"
string literal (Day 11) by explicitly calling out constants and literals.Emphasize Single Responsibility (Files):
- Current (
csharp.md
): “ALWAYS place each class, record, interface, or enum in its own file with a matching filename - this is a strict requirement, not just a preference”- Modification Suggestion (Add Emphasis): “STRICT REQUIREMENT: Adhere strictly to the ‘one class/record/interface/enum per file’ rule. When refactoring or reviewing code, actively check for violations and split files immediately if found.”
- Reasoning: Addresses the failure to suggest splitting
ChatHistory.cs
(Day 13) despite the existing rule. Adding active checking language might increase its priority.Reinforce Stub Implementation Constraint:
- Current (
default.md
): “Only fill in test stubs / todos if you are explicitly asked.”- Modification Suggestion (Add Emphasis): “STRICT: Do NOT implement
// TODO:
comments or fill test stubs unless explicitly instructed to do so in the current prompt.”- Reasoning: Addresses the repeated filling of test stubs even when not requested (Day 7, Day 9). Adding emphasis and tying it to the current prompt might help.
By adding these new conventions and strengthening existing ones, you aim to provide clearer, more prioritized instructions to the AI, hopefully reducing the need for corrective prompts and leading to more robust, maintainable, and correct code suggestions the first time.
Tokens: 168k sent, 1.1k received.
llm-history-to-post/core/Models/ChatHistory.cs
Add file to the chat? (Y)es/(N)o/(D)on’t ask again [Yes]: n
Verdict: ✅ And for the purposes of this year long adventure this seems like a great way to iteratively improve my conventions. Almost like training a junior dev.
Conclusion
An utter success! Looking forward to see what happens on the next month.