Skip to content

dnoliver/intel-ark-scrapper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
May 20, 2023
May 20, 2023
Jun 8, 2023
May 20, 2023
Jun 8, 2023
Jun 8, 2023
May 24, 2023
May 23, 2023
May 24, 2023
May 24, 2023
Jun 8, 2023
Jun 8, 2023
May 24, 2023
Jun 8, 2023

Repository files navigation

Intel Ark Scrapper

Scrapper Execution

time node index.js 2>&1

real    262m20.010s
user    0m0.295s
sys     0m0.983s

Schema Induction

node schema.js

Database Overview

Product Types

jq '.[].name' db.json

Product Brands

jq '.[0].products[].name' db.json

Product Series

jq '.[0].products[0].subproducts[].name' db.json

Products

jq '.[0].products[0].subproducts[0].skus[]."Product Name"' db.json

Specification Categories

jq '.[0].products[0].subproducts[0].skus[0].specs | keys' db.json

Data Loading

\copy ProductTypes FROM 'ProductTypes.csv' DELIMITER ',' CSV HEADER ENCODING 'UTF8';
\copy ProductBrands FROM 'ProductBrands.csv' DELIMITER ',' CSV HEADER ENCODING 'UTF8';
\copy ProductSeries FROM 'ProductSeries.csv' DELIMITER ',' CSV HEADER ENCODING 'UTF8';
\copy Products FROM 'Products.csv' DELIMITER ',' CSV HEADER ENCODING 'UTF8';
\copy ProductsSpecifications FROM 'ProductsSpecifications.csv' DELIMITER ',' CSV HEADER ENCODING 'UTF8';

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Packages

No packages published