About Future Scan
A research analysis tool I built and now use with a small group of strategists, researchers, and analysts
The project
Future Scan is my personal tool for staying current with AI/ML research. I built it to solve a problem I faced as a writer covering AI—hundreds of new papers are published every week, and manually tracking innovations, emerging trends, and research gaps was impossible.
The tool analyzes thousands of recent research papers from arXiv using advanced clustering algorithms to identify major themes and developments. Enter a research question, choose your methodology (thematic mapping, scoping review, or state of the art analysis), and the system retrieves relevant papers, clusters them into coherent themes, and generates comprehensive analysis reports—complete with interactive visualizations.
I work with a small group of strategists, researchers, and analysts who use Future Scan in their own work. It currently focuses exclusively on AI/ML research, covering October 2024 to August 2026 (187,837 papers) across key categories including Natural Language Processing, Computer Vision, Machine Learning, and Artificial Intelligence.
My goal is to continuously evolve Future Scan to better track new AI/ML innovations as they emerge from the research community.
How it works
Data Source: Papers are fetched from arXiv®'s open API, which provides access to titles, abstracts, authors, and metadata for millions of research papers.
Semantic Search: OpenAI's text-embedding-3-small model generates 1536-dimensional embeddings for paper abstracts, enabling intelligent semantic search using PostgreSQL's pgvector extension.
Clustering: Papers are grouped using cosine similarity via PostgreSQL's pgvector extension, automatically identifying distinct research themes based on semantic similarity.
Analysis: Claude Sonnet 4.6 analyzes each cluster to generate human-readable summaries, identify key papers, and explain technical approaches.
Visualization: Plotly creates interactive scatter plots showing how themes relate to each other in semantic space, using UMAP for dimensionality reduction.
What's next
I'm continuously evolving Future Scan to better track AI/ML innovations. Potential future directions:
Temporal trend tracking: Identify emerging vs. declining research themes over time
Cross-domain analysis: Track how AI/ML ideas move between different subfields
About me
I'm a writer and content strategist who learned systems design, machine learning, and data engineering by building my own AI-powered tools.
Future Scan builds on the technical foundation of an earlier project, StackDigest, a Django-based application featuring a vector database, semantic search, and ML-powered theme discovery across thousands of content assets. That work gave me hands-on experience with natural language processing, clustering algorithms, and vector similarity search (K-means, cosine similarity, pgvector)—all of which now power Future Scan's research trend analysis.
In my past roles as lead writer and content strategist, I've helped companies like Adobe, Confluent, and IBM develop content strategies, conduct market research, and write in-depth papers and reports for business and technical audiences.