Research, Design, and Planning (RDP) engagements at Atomic are a great way to de-risk a potential software project. Through technical investigation and human-centered design, we help organizations determine what – if any – custom software is right for the problem at hand. Typically, these engagements last eight to 12 weeks, and they are often the foundation of a successful, long-term software project. The amount of information one learns can be overwhelming, practically turning an Atomic team member into a subject matter expert (SME) overnight.
Not only is it challenging to absorb all this information so quickly, but it’s arguably more difficult to pin down the stakeholders and SMEs that answer questions and provide context around what you’re learning. We need to expedite that part of the work as much as possible without limiting our context or understanding, so we can get to providing real value.
So what do you do with all these questions when you have limited time with a client stakeholder or SME? One thing we’ve been exploring is building our subject matter experts with different AI tools, and it has proven to streamline a lot of the back and forth that comes with talking to an SME. I’ll explain how I’ve gone about doing this lately.
Get serious about data collection.
It’s important to curate your data sources to build an AI agent with the right information so it doesn’t just scan the internet and find whatever is out there. You should be borderline obsessive about collecting the data – record every conversation with different stakeholders and SMEs, or at least as much as is appropriate. You want this AI agent to become an expert in something, so provide it with whatever might be relevant.
Recently, we recorded an entire project kickoff workshop (six hours of content), software demos, and interviews with customers and key internal stakeholders. We also provided sales and training materials, which helped give the agent some context for how the current software works. The other helpful component was information about the broader industry. This client offers home healthcare management software, so I fed the AI agent resources about topics like Medicaid and private pay insurance agencies.
Having a variety of information we sourced meant we knew what the agent was becoming proficient in, but didn’t have to expend the time and mental capacity on becoming an expert in it ourselves. This not only resulted in time savings, but we could guarantee a level of quality and accuracy that would be uncertain if we just treated the chatbot like a Google search. As I started to gather the right data sources, it was time to experiment with different approaches, models, and tools.
Find the right tool(s) for the job.
There are so many options out there, so research and experiment to find what suits your needs. I tried different models, applications, and prompt styles to get something that started to work well. And honestly, I could keep tinkering. The options are endless, so timebox your efforts and get something that works well enough for the task at hand.
After some trial and error, I landed on using Claude 3 Opus. The context limits are impressive and higher than other models. This means you can feed it more data, and have it remember that data for longer (before you have to adjust the context window). I would often keep the context set to the entire conversation for as long as possible but eventually would have to set it to the last 10-15 messages. At that point, I might even have to re-feed it the same data from before. Either way, it was still much better at processing large quantities of information, and the responses felt more accurate, insightful, and well-written compared to other models.
In addition to finding the right language, having the right user interface (UI) also made a big difference in my workflow. I tried MindMac and Typing Mind, which both had their pros and cons regarding usability and simplicity. However, Typing Mind became my go-to UI, largely because of its ability to scan web pages on your behalf. That became critical in training the agent to understand our clients’ offering, their competition, and the broader market they operate. Then once the data and tools were in place, I could focus on how to prompt the agent to get the best results.
Be creative with your prompts.
It makes all the difference in how you prompt the agent when trying to produce relevant, valuable insights. It will analyze data differently depending on how you direct it. I often knew what I wanted to get out of the information I was providing, so I had a rough idea of what the result should look like. I just needed to save time by not having to go find and analyze all of the information by myself. Strategically prompting it for the results I needed was a game changer.
For example, I would take a transcript from an interview I conducted and already know what some key takeaways were. I remembered parts of the conversation or certain things someone said that I needed to look at closer. I could upload a transcript and ask the agent to extract all the relevant quotes about this feature or that customer’s pain point. Once I provided it with some guidelines, it analyzed the data for me and returned what I needed in minutes, saving me an hour or two each time. I could then spend my time taking that information and doing something with it, rather than just draining all my brain power on data analysis.
Build your own SME.
Finding the right software solution is no walk in the park, but the real challenge often lies in analyzing the current state of a business or product ecosystem and framing the problem that needs to be solved. At one point or another, this requires subject matter expertise in the relevant domain(s). Since clients don’t come to us to gain another expert in their field, but rather to build software solutions, we need to streamline the understanding process where possible.
Using various AI tools to understand large amounts of information is a great way to quickly gain SME capabilities on your team. We still need domain expertise to build a valuable product. The difference now is that we can save our time, and that of our clients, by building AI subject matter experts to assist in the knowledge transfer. So rather than absorbing information all day, we can do what we do best – build great products.