Research Gap Analysis – July 7, 2026
Honest Statistics
| Metric |
Value |
| arXiv queries executed |
20 |
| arXiv results per query |
50 |
| Total real papers saved |
1000 |
| Semantic Scholar API queries attempted |
10 |
| Semantic Scholar results obtained |
0 |
| Thesis titles generated |
210 |
| Domain analysis files |
4 (NLP, Cybersecurity, Vision, Surveys) |
File Inventory
| File |
Description |
analyses/ |
20 raw arXiv query exports (50 papers each) |
nlp-analysis.md |
NLP domain analysis with gaps and paper references |
cybersecurity-analysis.md |
Cybersecurity domain analysis |
vision-analysis.md |
Computer vision domain analysis |
surveys-analysis.md |
Survey/benchmark domain analysis |
titles.md |
210 thesis titles derived from real paper gaps |
scrape_arxiv.js |
arXiv API scraper script |
scrape_s2.js |
Semantic Scholar API scraper (rate-limited) |
Limitations & Caveats
- arXiv only: All 1,000 papers come from arXiv. Conference papers (ACL, NeurIPS, CVPR) indexed only on Semantic Scholar are not included.
- Semantic Scholar API blocked: The Semantic Scholar API returned HTTP 429 (Too Many Requests) consistently, even after WARP VPN cycling. WARP IP ranges appear to be blocked. Attempted: 3 retries per query with 10-40s exponential backoff. No S2 data was obtained.
- Query bias: Results reflect the specific 20 search queries chosen. Alternative queries would surface different papers.
- Relevance filtering: arXiv API returns results sorted by relevance, but some papers may be only tangentially related to the query topic.
- Date range: arXiv API default sorting is by relevance, not date. Papers range from ~2010 to July 2026.
- Indonesian focus: The analysis emphasizes Indonesian-language and developing-country gaps, which may not align with mainstream NLP/CV/security research priorities.
Instructions for Reproduction
# 1. Ensure WARP is connected (or use direct connection)
# 2. Run arXiv scraper (20 queries, ~60 seconds with delays)
node scrape_arxiv.js
# 3. Run Semantic Scholar scraper (if rate limits permit)
node scrape_s2.js
# 4. Regenerate output files
node generate_outputs.js
Notes for Thesis Use
- Each title in
titles.md references a specific gap visible in the scraped literature. The gap is stated after the title.
- Domain analysis files reference real papers by arXiv ID for verifiability.
- Before finalizing any title, verify the gap by checking the corresponding papers in
analyses/.
- The
titles.md file organizes titles into 5 categories: NLP (30), Cybersecurity (30), Vision (30), LLM Evaluation (20), Cross-Domain/Surveys (20), Hybrid/Applied (80).
Generated on July 7, 2026 | Data source: arXiv API | S2: rate-limited