About Us

Our team and technology crawl the internet to index, organize, and summarize English-language Jewish media. Paid supporters help fund our operations, bringing Jewish media to wider audiences, and also receive a few bonuses: concise summaries of all articles, Shabbat-friendly printable collections, and search tools covering our full archive.

Team

• Jonathan Abbett
• Matt Sienkiewicz

Cairo, our web crawler

JLDR is driven primarily by Cairo, a focused web crawler named for the Cairo Geniza. Unlike traditional crawlers such as GoogleBot, a focused crawler is designed to be highly selective. We programmed Cairo to crawl websites that specifically publish Jewish content, or to crawl general media sites just for individual links about Jewish topics.

We created JLDR because we love Jewish media and want it to be easier to find. We want the websites we crawl to see us as a partner in driving more visitors to their Jewish content.

Cairo looks for links to articles, essays, reports, podcasts, and videos. When it finds an item of interest, Cairo stores the item's metadata (like title, author, publication date) and queues the item for curation.

Cairo curates an item by generating a brief summary, generating a set of topic-based tags, and categorizing the item.

Found a problem with Cairo? We try hard to make Cairo reliable and respectful of the websites it crawls. If you notice any unusual activity from Cairo, you can reach us at hello@jldr.org. Your feedback helps us improve.

Want to block Cairo? If you have a problem or concern, we much prefer the chance to address it, but if you need to block Cairo, we respect the robots.txt exclusion list. Cairo's user-agent is: CairoCrawler