Head of Cyber, KPMG Singapore | Co-chair, Singapore Artificial Intelligence Technical Committee (AITC) | Certified AI Ethics & Governance (Expert) | CRGAIG PROFESSIONAL (Professional)
One of the challenges of keeping pace with the rapid development of AI is reading up on new developments, connecting the ideas with your experience, and the ability to retrieve the information when required. The process I had for this was too time consuming, which often led to missing out on some insights that helps my knowledge growth. I used to spend time scrolling through the daily arXiv email updates, scanning for abstracts that might be of interest to me. I would then open them up in tabs and hoping that I would find time to quickly scan through the paper to see if there are areas of interest that I am keen in delving deeper into. But finding the time was always difficult especially with nearly 80% of the daily feeds (which can go to the range of 150 papers or more per day) being irrelevant. Over the past week, I have tried out this new automated workflow combining API calls to arXiv and Notion along with LangChain to build two pipelines to recommend papers to read along with a summary for those that I then further filter. It works on my local Llama 3.1 though I’ve found it more cost effective to just use the OpenAI GPT4o-mini model. The key benefit of this workflow is that nothing leaves the consumption process. I no longer have to spend time manually scanning, copying to some place for reading, and wondering where I kept the notes. Everything is centralized within the Notion database including the AI-generated summaries and my own notes and connections to other ideas. With the script automated to run every morning, the 150+ articles a day goes through a filtration process with increasing levels of details. The summarization picks up a few key highlights from the papers: • What were the problems the researchers were trying to address? • What are the research questions formulated to address this? • What are the key findings and insights? • What future work do they envision? • [Future roadmap] What are the other papers they referenced? By keeping everything in my Notion knowledge graph, I can retrieve my connected notes when I am preparing for a presentation or an engagement. As a future work, this can be easily extended to cover other sources of information to keep up to date with lesser friction while staying within a searchable knowledge graph. If you have similar needs, happy to discuss further and share the MVP code with you on request, along with what I learnt while building this. #Notion #AI #LangChain