Lab 3 Synthetic Dataset Master Index
This static site contains 50 synthetic HTML test documents generated for programming assignment Lab 3 (CCPS 842 – Information Retrieval and Web Search) by Sohrab Soltani. The dataset is divided into two distinct experimental test suites: near-duplicate detection (Part 1) and web spam / content quality detection (Part 2).
Part 1: Coffee-Maker Products (Near-Duplicate Detection)
Consists of 10 product pages split across 2 fictional brands (Apex Brewing Co. and Zenith Coffee Tech). Designed to evaluate shingling algorithms and Jaccard similarity bounds.
| Pair # | Product Category | Company A (Original) | Company B (Counterpart) | Similarity Type | Target Jaccard Bound |
|---|---|---|---|---|---|
| 1 | Pour-Over Station | Apex Artisanal Pour-Over | Zenith Pour-Master | Near-Duplicate | > 80% (k=3) |
| 2 | 12-Cup Drip Brewer | Apex Drip Pro 12-Cup | Zenith Drip Elite 12-Cup | Near-Duplicate | > 80% (k=3) |
| 3 | Espresso Machine | Apex Espresso Max | Zenith Espresso Vibe | Near-Duplicate | > 80% (k=3) |
| 4 | Cold Brew Steeper | Apex Cold Craft Steeper | Zenith Cold-Chill Brewer | Paraphrased Duplicate | < 5% (k=3) |
| 5 | Single-Serve Pod | Apex Pod Swift Express | Zenith Pod-Express Compact | Paraphrased Duplicate | < 5% (k=3) |
Part 2: Web Spam & Quality Evaluation (40 Pages)
Consists of 20 base topic pages and 20 cloaking counterpart pages (_cloaked_view.html) constructed to test heuristic algorithms for keyword stuffing (Q6), hidden text (Q7), cloaking (Q8), doorway pages (Q9), link spam (Q10), text quality (Q11), keyword density (Q12), and link diversity (Q13).
| # | Topic Page (Crawler View) | Cloaked View (User View) | Keyword (Count) | Quality Tier | Links (Tot/Uniq) | Doorway | Hidden Text | Cloaking Pair |
|---|---|---|---|---|---|---|---|---|
| 1 | Running Shoes Buying Guide | User View | shoes Neg (1) | Tier 2 | 12 / 1 | JS | No | Different |
| 2 | Rome Travel Guide | User View | rome Pos (6) | Tier 3 | 12 / 1 | JS | Yes | Different |
| 3 | Sourdough Bread Baking | User View | sourdough Neg (2) | Tier 4 | 12 / 2 | JS | No | Identical |
| 4 | Mountain Biking Basics | User View | biking Pos (8) | Tier 5 | 12 / 2 | JS | Yes | Identical |
| 5 | Personal Budgeting Basics | User View | budget Neg (3) | Tier 1 | 12 / 3 | JS | No | Different |
| 6 | Houseplant Care Guide | User View | houseplants Pos (10) | Tier 2 | 12 / 3 | META | Yes | Identical |
| 7 | Smartphone Photography Tips | User View | photography Neg (4) | Tier 3 | 12 / 4 | META | No | Identical |
| 8 | Online Python Programming Courses | User View | python Pos (13) | Tier 4 | 12 / 4 | META | Yes | Different |
| 9 | Electric Vehicles Overview | User View | electric Neg (5) | Tier 5 | 12 / 5 | META | No | Different |
| 10 | Yoga for Beginners | User View | yoga Pos (16) | Tier 1 | 12 / 5 | META | Yes | Identical |
| 11 | Local Farmers Markets | User View | farmers Neg (1) | Tier 2 | 10 / 6 | None | No | Identical |
| 12 | Kitchen Remodeling Ideas | User View | kitchen Pos (6) | Tier 3 | 10 / 6 | None | Yes | Different |
| 13 | Dog Training Basics | User View | dog Neg (2) | Tier 4 | 10 / 7 | None | No | Identical |
| 14 | Solar Panel Installation Guide | User View | solar Pos (8) | Tier 5 | 10 / 7 | None | Yes | Different |
| 15 | History of Jazz Music | User View | jazz Neg (3) | Tier 1 | 10 / 8 | None | No | Identical |
| 16 | Beginner's Guide to Chess | User View | chess Pos (10) | Tier 2 | 10 / 8 | None | Yes | Identical |
| 17 | Sustainable Fashion Guide | User View | fashion Neg (4) | Tier 3 | 10 / 9 | None | No | Different |
| 18 | Freshwater Aquarium Setup | User View | aquarium Pos (13) | Tier 4 | 10 / 9 | None | Yes | Different |
| 19 | Basics of Cryptocurrency Investing | User View | crypto Neg (5) | Tier 5 | 10 / 10 | None | No | Different |
| 20 | Beginner Woodworking Projects | User View | woodworking Pos (16) | Tier 1 | 10 / 10 | None | Yes | Identical |