Gen AI Challenges & Limitations.

What are the challenges and limitations of Gen AI Technology or Approach?

1. System Instructions (System Prompting) play an important role!

I'd found that System_Instructions strongly impacts Model and Agent behavior, Chat flow and the quality of the outcome/responses. It plays an important role in Model effectiveness or what is considered a "GOOD" response.

• With a basic system_instructions, the challenge is that the Model sometimes don't "remember" it knows the table schema even though it is told to use the table_schema. When it "forgets", it says it does not know that information, then asks the user to provide that schema, even though generally the user would not have that system-level information.

• To solve that problem, I "tweeked" the system_instructions so the model will have clearer and more specific instructions what to do when it has an "undeterministic" behavior such as it forgets what it should already know.

• I'd found that the more specific the instructions it does help the output/response from the model to overcome it's non-determonistic behavior, such as to then remember to look up restaurant information in the table scheme that it already knows.

• However, if I get too rigid in my system instructions, such as when I specifically ask the model to "not ask the user for table schema" when it forgots on multi-turn chat, then it seems to cut down on multi-turn chat which then prevents it from successfully investigating and displaying the Chat-History (did it have an attitude?!). So tweeking the System_Instructions JUST ENOUGH seems to be a sweet spot in deterministic/non-deterministic behavior to accomplish the task.

2. Short memory!

Another problem is that it forgets which restaurants it had already recommended to the user on multi-turn chat, which feels like a strange chat conversation when it just recommended the restaurants but upon further queries about the recommendations, that it suddenly didn't remember what it just recommended! So tweeking the System_Instructions "to remember the recommendations" worked because I told it that it was a "smart agent" seemed to help the forgetfulness, but just like #1 above, it also caused another problem that it limits the Chat History so even though it no longer forgets its recommendation, but then the investigation fails to display multi-turn chat! Somehow it does not seem to have enough chat history to answer, then investigation crashes on "Function Response". When removing that specific system instruction then the multi-turn chat investigation works again! So the sweet spot seems to be using System_Instruction to not restrict it too much, yet provide enough guidance.

3. Model erroneously uses it's own pre-trained knowledge

Sometimes the model will use it's pre-trained knowledge to recommend restaurants that are not in the curated local database. When it does that, the restaurants it recommended were highly popular restaurants with a very high price of 100USD per meal, which is not part of the criteria, nor in the database. When I asked how it got this answer, it does not answer me :-)

4. The specific Gen AI Model used makes a difference and beware it can Cost!

Only the later Gen AI models can do live check of current weather conditions! But using the Gemini Advanced 2.5 Pro (Experimental) model had a limitation to the free-tier!

After some runs, it prevented me from doing more runs due to exceeding the free-tier. I had to enter a credit card number to continue, which I did NOT want to do for a public Capstone Project. Luckily I solved the issue by using an earlier Gen AI model (Gemini 2.0-flash) in the Google Search Grounding part of the Capstone Project that can luckily still do a real-time query for current weather conditions, which I needed to solve my “Where to eat” use case!