Any suggestions for ai tool to use for the following?

Upload CVS file with about 8000 rows and 3 columns.

The objective is for the ai tool to research the manufacturer #model and return an ai generated, formatted description into a column for each manufacturer and model number that is used in our MRP system (we have a bunch of bad and inconsistent descriptions we want to replace.

Many - 80% - of items are easy and will be returned in internet searches. When providing instructions / learning, grok did well, chatgpt probably better, I could type in any part number and get a consistently formatted, accurate description returned one at a time with no problem.

gemini was unstable and gave an error most of the time on anything I put in.

So the problem with grok was it could not return a csv file at all (trying every method possible). It could not process more than 200 rows at a time. Eventually, the chat got too long and stopped responding and trying to continue it was basically a start over. The longer you interfaced with it, the dumber it got.

Chatgpt was promising, but it is processing about 1 item every 3-5 minutes which is slower than me manually looking all these up and typing them in. It also has a short runtime that stops the process before writing even 5 descriptions to the csv. I solved this by having it be a watchdog and constantly save, index and give me a link for the file. Eventually this chat had gotten too long and stopped working before.

The more complicated part of this is that some items have long, configured part #s that I have uploaded the manual / configurator for. It seems to do OK on these one at a time, but ask it to do 5 and it melts down. I have also told it to just use the existing description if it has trouble finding information for an item.

The first 5 minutes is easy and results are phenomenal. After that, both tools have been disasters at trying to find a way to process the 8000 items.

Any suggestions?

Cursor, and use Sonnet-4 (in Cursor) to write you a python script that will do what you want. I haven't used the llm api's for doing search, so I don't know how difficult that is, but I assume it is relatively easy. The rest of the requirements should be no problem.

Thank you for suggestion.

After posting this, I circled back to CoPilot and it seems to have absolutely nailed this. I tried it a couple of months ago and it crashed and burned quickly.

I woke up this morning and 7000 of the items are processed.

Now after Co-pilot nailed everything to my satisfaction, it becomes impossible to get the data created out of it... same issue as grok and chatgpt.

They are all an endless cycle of getting to perfection, spitting out data and hitting runtime issues limits when providing the data. All of them. They can't handle outputting large batches so you give it smaller batches and it doesnt take long to experience runtime interruptions that make it crater and forget everything.

Breaking it up into small pieces makes you have to sit and constantly remind it to output data and ends in failure.

Co-pilot and grok wont write into the csv file, chatgpt will. But they ultimately all crater.

Have you tried NotebookLM?

bthotugigem05 said:

---

Have you tried NotebookLM?

---

Have not but will take a look thank you.

BadAzzBohemian said:

---

Now after Co-pilot nailed everything to my satisfaction, it becomes impossible to get the data created out of it... same issue as grok and chatgpt.

They are all an endless cycle of getting to perfection, spitting out data and hitting runtime issues limits when providing the data. All of them. They can't handle outputting large batches so you give it smaller batches and it doesnt take long to experience runtime interruptions that make it crater and forget everything.

Breaking it up into small pieces makes you have to sit and constantly remind it to output data and ends in failure.

Co-pilot and grok wont write into the csv file, chatgpt will. But they ultimately all crater.

---

If you buy a license for Copilot for M365, you can use Copilot directly in Excel, and it will change your life.

Rex Racer said:

---

BadAzzBohemian said:

---

Now after Co-pilot nailed everything to my satisfaction, it becomes impossible to get the data created out of it... same issue as grok and chatgpt.

They are all an endless cycle of getting to perfection, spitting out data and hitting runtime issues limits when providing the data. All of them. They can't handle outputting large batches so you give it smaller batches and it doesnt take long to experience runtime interruptions that make it crater and forget everything.

Breaking it up into small pieces makes you have to sit and constantly remind it to output data and ends in failure.

Co-pilot and grok wont write into the csv file, chatgpt will. But they ultimately all crater.

---

If you buy a license for Copilot for M365, you can use Copilot directly in Excel, and it will change your life.

---

This.

Also look into agents. They should be able to take inputs / kick out results.

I have realized the issue. All AI requiring internet AI searches prevent running batch processes to ensure processing availability is protected.

It will do 10 at a time perfectly, but no more. If batches of more than 10 are done, it corrupts itself by design when asking for the next 10 or when asking it to run a batch of 20.

I will try a license for co-pilot, but what I am doing requires internet lookup of part #s. AI will process my 8000 row spreadsheet nearly instantly to do something that doesn't require anything beyond a superficial internet search.

I have a ChatGPT sub and trying to do anything with a CSV file over 100 lines is cumbersome as hell.

Is there a way to set a trigger to start the search and a limit on how many to run? If so, use something like Zapier or Make to send you a trigger on a schedule, run a few then stop. This will allow you to get the results that you need but not have to run them continuously.

n_touch said:

---

Is there a way to set a trigger to start the search and a limit on how many to run? If so, use something like Zapier or Make to send you a trigger on a schedule, run a few then stop. This will allow you to get the results that you need but not have to run them continuously.

---

I would say no. This is a never ending battle of even if you get it to process 10 or so at a time, the more triggers it receives the more it starts doing everything possible to get you to stop. It starts introducing problems and doing the stupidest things imaginable because it doesnt want to run 200 items much less 8000 items.

This is why I recommended Cursor. It is a DIY solution, but Claude Sonnet will do all the heavy lifting, and it will even test it and makes sure it can run on your system. It will empower you to build out more complex systems without the constraints the out-of-the-box solutions will impose.