The arXiv Scout

The 100 Days of Making AI is not just about vibe coding a bunch of projects.  Yes it's partially that, but I also want to explore AI capabilities including agents, evaluation and fine tuning.

Today I built my first AI agent.  Something that is actually agentic and may infact be somewhat useful.  The arXiv Scout is a Python app that takes a research topic provided by the user and goes to arXiv.org to find any relevant papers.  Papers are reviewed and ranked, and then a first and second loop are executed to expand the search.  This is all handeld by the agent.

The first query returns 25 items, which are then ranked based on keyword an recency.  The the top five articles are then sent to an LLM to summaryize AND suggest 1-2 follow up queries.  New results from the updated query are then deduplicated and rated again to generate a top five.  Once a sufficent amount of resources are found, the LLM then synthesizes and creates a markdown and json output.  I'm using 

I havent fully tested it to see how well it performs, but I'm using gptoss-120b and on a few queries and iterations it seemed to return meaninful information.  And most importantly, the articles actually exist!  No hallucinations!

Code can be found on Github