{“content”:”---\nname: arxiv\ndescription: Search and retrieve academic papers from arXiv using their free REST API. No API key needed. Search by keyword, author, category, or ID. Combine with web_extract or the ocr-and-documents skill to read full paper content.\nversion: 1.0.0\nauthor: Hermes Agent\nlicense: MIT\nmetadata:\n hermes:\n tags: [Research, Arxiv, Papers, Academic, Science, API]\n related_skills: [ocr-and-documents]\n---\n\n# arXiv Research\n\nSearch and retrieve academic papers from arXiv via their free REST API. No API key, no dependencies — just curl.\n\n## Quick Reference\n\n| Action | Command |\n|--------|---------|\n| Search papers | curl \"https://export.arxiv.org/api/query?search_query=all:QUERY&max_results=5\" |\n| Get specific paper | curl \"https://export.arxiv.org/api/query?id_list=2402.03300\" |\n| Read abstract (web) | web_extract(urls=[\"https://arxiv.org/abs/2402.03300\"]) |\n| Read full paper (PDF) | web_extract(urls=[\"https://arxiv.org/pdf/2402.03300\"]) |\n\n## Searching Papers\n\nThe API returns Atom XML. Parse with grep/sed or pipe through python3 for clean output.\n\n### Basic search\n\nbash\ncurl -s \"https://export.arxiv.org/api/query?search_query=all:GRPO+reinforcement+learning&max_results=5\"\n\n\n### Clean output (parse XML to readable format)\n\nbash\ncurl -s \"https://export.arxiv.org/api/query?search_query=all:GRPO+reinforcement+learning&max_results=5&sortBy=submittedDate&sortOrder=descending\" | python3 -c \"\nimport sys, xml.etree.ElementTree as ET\nns = {'a': 'http://www.w3.org/2005/Atom'}\nroot = ET.parse(sys.stdin).getroot()\nfor i, entry in enumerate(root.findall('a:entry', ns)):\n title = entry.find('a:title', ns).text.strip().replace('\\n', ' ')\n arxiv_id = entry.find('a:id', ns).text.strip().split('/abs/')[-1]\n published = entry.find('a:published', ns).text[:10]\n authors = ', '.join(a.find('a:name', ns).text for a in entry.findall('a:author', ns))\n summary = entry.find('a:summary', ns).text.strip()[:200]\n cats = ', '.join(c.get('term') for c in entry.findall('a:category', ns))\n print(f'{i+1}. [{arxiv_id}] {title}')\n print(f' Authors: {authors}')\n print(f' Published: {published} | Categories: {cats}')\n print(f' Abstract: {summary}...')\n print(f' PDF: https://arxiv.org/pdf/{arxiv_id}')\n print()\n\"\n\n\n## Search Query Syntax\n\n| Prefix | Searches | Example |\n|--------|----------|---------|\n| all: | All fields | all:transformer+attention |\n| ti: | Title | ti:large+language+models |\n| au: | Author | au:vaswani |\n| abs: | Abstract | abs:reinforcement+learning |\n| cat: | Category | cat:cs.AI |\n| co: | Comment | co:accepted+NeurIPS |\n\n### Boolean operators\n\n\n# AND (default when using +)\nsearch_query=all:transformer+attention\n\n# OR\nsearch_query=all:GPT+OR+all:BERT\n\n# AND NOT\nsearch_query=all:language+model+ANDNOT+all:vision\n\n# Exact phrase\nsearch_query=ti:\"chain+of+thought\"\n\n# Combined\nsearch_query=au:hinton+AND+cat:cs.LG\n\n\n## Sort and Pagination\n\n| Parameter | Options |\n|-----------|---------|\n| sortBy | relevance, lastUpdatedDate, submittedDate |\n| sortOrder | ascending, descending |\n| start | Result offset (0-based) |\n| max_results | Number of results (default 10, max 30000) |\n\nbash\n# Latest 10 papers in cs.AI\ncurl -s \"https://export.arxiv.org/api/query?search_query=cat:cs.AI&sortBy=submittedDate&sortOrder=descending&max_results=10\"\n\n\n## Fetching Specific Papers\n\nbash\n# By arXiv ID\ncurl -s \"https://export.arxiv.org/api/query?id_list=2402.03300\"\n\n# Multiple papers\ncurl -s \"https://export.arxiv.org/api/query?id_list=2402.03300,2401.12345,2403.00001\"\n\n\n## BibTeX Generation\n\nAfter fetching metadata for a paper, generate a BibTeX entry:\n\n{% raw %}\nbash\ncurl -s \"https://export.arxiv.org/api/query?id_list=1706.03762\" | python3 -c \"\nimport sys, xml.etree.ElementTree as ET\nns = {'a': 'http://www.w3.org/2005/Atom', 'arxiv': 'http://arxiv.org/schemas/atom'}\nroot = ET.parse(sys.stdin).getroot()\nentry = root.find('a:entry', ns)\nif entry is None: sys.exit('Paper not found')\ntitle = entry.find('a:title', ns).text.strip().replace('\\n', ' ')\nauthors = ' and '.join(a.find('a:name', ns).text for a in entry.findall('a:author', ns))\nyear = entry.find('a:published', ns).text[:4]\nraw_id = entry.find('a:id', ns).text.strip().split('/abs/')[-1]\ncat = entry.find('arxiv:primary_category', ns)\nprimary = cat.get('term') if cat is not None else 'cs.LG'\nlast_name = entry.find('a:author', ns).find('a:name', ns).text.split()[-1]\nprint(f'@article{{{last_name}{year}_{raw_id.replace(\\\".\\\", \\\"\\\")},')\nprint(f' title = {{{title}}},')\nprint(f' author = {{{authors}}},')\nprint(f' year = {{{year}}},')\nprint(f' eprint = {{{raw_id}}},')\nprint(f' archivePrefix = {{arXiv}},')\nprint(f' primaryClass = {{{primary}}},')\nprint(f' url = {{https://arxiv.org/abs/{raw_id}}}')\nprint('}')\n\"\n\n{% endraw %}\n\n## Reading Paper Content\n\nAfter finding a paper, read it:\n\n\n# Abstract page (fast, metadata + abstract)\nweb_extract(urls=[\"https://arxiv.org/abs/2402.03300\"])\n\n# Full paper (PDF → markdown via Firecrawl)\nweb_extract(urls=[\"https://arxiv.org/pdf/2402.03300\"])\n\n\nFor local PDF processing, see the ocr-and-documents skill.\n\n## Common Categories\n\n| Category | Field |\n|----------|-------|\n| cs.AI | Artificial Intelligence |\n| cs.CL | Computation and Language (NLP) |\n| cs.CV | Computer Vision |\n| cs.LG | Machine Learning |\n| cs.CR | Cryptography and Security |\n| stat.ML | Machine Learning (Statistics) |\n| math.OC | Optimization and Control |\n| physics.comp-ph | Computational Physics |\n\nFull list: https://arxiv.org/category_taxonomy\n\n## Helper Script\n\nThe scripts/search_arxiv.py script handles XML parsing and provides clean output:\n\nbash\npython scripts/search_arxiv.py \"GRPO reinforcement learning\"\npython scripts/search_arxiv.py \"transformer attention\" --max 10 --sort date\npython scripts/search_arxiv.py --author \"Yann LeCun\" --max 5\npython scripts/search_arxiv.py --category cs.AI --sort date\npython scripts/search_arxiv.py --id 2402.03300\npython scripts/search_arxiv.py --id 2402.03300,2401.12345\n\n\nNo dependencies — uses only Python stdlib.\n\n---\n\n## Semantic Scholar (Citations, Related Papers, Author Profiles)\n\narXiv doesn’t provide citation data or recommendations. Use the Semantic Scholar API for that — free, no key needed for basic use (1 req/sec), returns JSON.\n\n### Get paper details + citations\n\nbash\n# By arXiv ID\ncurl -s \"https://api.semanticscholar.org/graph/v1/paper/arXiv:2402.03300?fields=title,authors,citationCount,referenceCount,influentialCitationCount,year,abstract\" | python3 -m json.tool\n\n# By Semantic Scholar paper ID or DOI\ncurl -s \"https://api.semanticscholar.org/graph/v1/paper/DOI:10.1234/example?fields=title,citationCount\"\n\n\n### Get citations OF a paper (who cited it)\n\nbash\ncurl -s \"https://api.semanticscholar.org/graph/v1/paper/arXiv:2402.03300/citations?fields=title,authors,year,citationCount&limit=10\" | python3 -m json.tool\n\n\n### Get references FROM a paper (what it cites)\n\nbash\ncurl -s \"https://api.semanticscholar.org/graph/v1/paper/arXiv:2402.03300/references?fields=title,authors,year,citationCount&limit=10\" | python3 -m json.tool\n\n\n### Search papers (alternative to arXiv search, returns JSON)\n\nbash\ncurl -s \"https://api.semanticscholar.org/graph/v1/paper/search?query=GRPO+reinforcement+learning&limit=5&fields=title,authors,year,citationCount,externalIds\" | python3 -m json.tool\n\n\n### Get paper recommendations\n\nbash\ncurl -s -X POST \"https://api.semanticscholar.org/recommendations/v1/papers/\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"positivePaperIds\": [\"arXiv:2402.03300\"], \"negativePaperIds\": []}' | python3 -m json.tool\n\n\n### Author profile\n\nbash\ncurl -s \"https://api.semanticscholar.org/graph/v1/author/search?query=Yann+LeCun&fields=name,hIndex,citationCount,paperCount\" | python3 -m json.tool\n\n\n### Useful Semantic Scholar fields\n\ntitle, authors, year, abstract, citationCount, referenceCount, influentialCitationCount, isOpenAccess, openAccessPdf, fieldsOfStudy, publicationVenue, externalIds (contains arXiv ID, DOI, etc.)\n\n---\n\n## Complete Research Workflow\n\n1. Discover: python scripts/search_arxiv.py \"your topic\" --sort date --max 10\n2. Assess impact: curl -s \"https://api.semanticscholar.org/graph/v1/paper/arXiv:ID?fields=citationCount,influentialCitationCount\"\n3. Read abstract: web_extract(urls=[\"https://arxiv.org/abs/ID\"])\n4. Read full paper: web_extract(urls=[\"https://arxiv.org/pdf/ID\"])\n5. Find related work: curl -s \"https://api.semanticscholar.org/graph/v1/paper/arXiv:ID/references?fields=title,citationCount&limit=20\"\n6. Get recommendations: POST to Semantic Scholar recommendations endpoint\n7. Track authors: curl -s \"https://api.semanticscholar.org/graph/v1/author/search?query=NAME\"\n\n## Rate Limits\n\n| API | Rate | Auth |\n|-----|------|------|\n| arXiv | ~1 req / 3 seconds | None needed |\n| Semantic Scholar | 1 req / second | None (100/sec with API key) |\n\n## Notes\n\n- arXiv returns Atom XML — use the helper script or parsing snippet for clean output\n- Semantic Scholar returns JSON — pipe through python3 -m json.tool for readability\n- arXiv IDs: old format (hep-th/0601001) vs new (2402.03300)\n- PDF: https://arxiv.org/pdf/{id} — Abstract: https://arxiv.org/abs/{id}\n- HTML (when available): https://arxiv.org/html/{id}\n- For local PDF processing, see the ocr-and-documents skill\n\n## ID Versioning\n\n- arxiv.org/abs/1706.03762 always resolves to the latest version\n- arxiv.org/abs/1706.03762v1 points to a specific immutable version\n- When generating citations, preserve the version suffix you actually read to prevent citation drift (a later version may substantially change content)\n- The API <id> field returns the versioned URL (e.g., http://arxiv.org/abs/1706.03762v7)\n\n## Withdrawn Papers\n\nPapers can be withdrawn after submission. When this happens:\n- The <summary> field contains a withdrawal notice (look for “withdrawn” or “retracted”)\n- Metadata fields may be incomplete\n- Always check the summary before treating a result as a valid paper\n”}