Presumably a document query system, one of the actually useful reasons to use an LLM.
Having it trawl through thousands of pages of NASA and government files to answer regulatory questions is probably legitimately helpful, even if all it does is point you towards the right pages for a human to review.
That makes sense. Like you, I’ve generally found that LLMs are incredibly useful for certain, highly specific things but people (CEOs especially) need to understand their limitations.
When it first came out, I purposely used ChatGPT on a trip to evaluate it. I was in a historic city on a business trip where I stayed an extra few days so I was traveling alone. It was good at being a tour guide. Obviously, I could have researched everything and read guidebooks but I was focused on my work stuff. Being able to ask follow-up questions and have a conversation was a real improvement over traditional search.
That’s obviously a limited use case where I was asking questions that could have been answered in traditional ways but I found that to be a good consumer use case. It knew details that wouldn’t necessarily be in a Wikipedia article or Guidebook that would take me 15 Google searches to answer. Just my own little curiosity questions about an old building or whatever. I cross-checked things later and it didn’t hallucinate. Obviously, a very limited use case but it was good at it.
This is the one I’m waiting the most from the LLM hype. It would be a massive benefit for companies around the world (mine included) if they could just dump their documentation in all shapes and flavours into a model and have it parse standardized documents for you.
But the generic OpenAI/Copilot models aren’t reliable enough just yet, hallucinations and made up data just doesn’t go with that. I’m not even sure if those models are ever capable on such task alone, maybe it needs additional component which checks the facts from originals or something to make it actually useful.
I created a relatively complex board/card game and just shoved the rulebook PDF into an LLM, so it’d be easy to answer people’s questions with exact references to cards and rulebook pages. So far it’s successfully corrected ME twice on specific rulings. It’s actually quite useful in this regard.
Presumably a document query system, one of the actually useful reasons to use an LLM.
Having it trawl through thousands of pages of NASA and government files to answer regulatory questions is probably legitimately helpful, even if all it does is point you towards the right pages for a human to review.
That makes sense. Like you, I’ve generally found that LLMs are incredibly useful for certain, highly specific things but people (CEOs especially) need to understand their limitations.
When it first came out, I purposely used ChatGPT on a trip to evaluate it. I was in a historic city on a business trip where I stayed an extra few days so I was traveling alone. It was good at being a tour guide. Obviously, I could have researched everything and read guidebooks but I was focused on my work stuff. Being able to ask follow-up questions and have a conversation was a real improvement over traditional search.
That’s obviously a limited use case where I was asking questions that could have been answered in traditional ways but I found that to be a good consumer use case. It knew details that wouldn’t necessarily be in a Wikipedia article or Guidebook that would take me 15 Google searches to answer. Just my own little curiosity questions about an old building or whatever. I cross-checked things later and it didn’t hallucinate. Obviously, a very limited use case but it was good at it.
This is the one I’m waiting the most from the LLM hype. It would be a massive benefit for companies around the world (mine included) if they could just dump their documentation in all shapes and flavours into a model and have it parse standardized documents for you.
But the generic OpenAI/Copilot models aren’t reliable enough just yet, hallucinations and made up data just doesn’t go with that. I’m not even sure if those models are ever capable on such task alone, maybe it needs additional component which checks the facts from originals or something to make it actually useful.
I created a relatively complex board/card game and just shoved the rulebook PDF into an LLM, so it’d be easy to answer people’s questions with exact references to cards and rulebook pages. So far it’s successfully corrected ME twice on specific rulings. It’s actually quite useful in this regard.