Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add features url for photo post for next update? #31

Open
jackingpanda opened this issue Jan 3, 2025 · 5 comments
Open

Add features url for photo post for next update? #31

jackingpanda opened this issue Jan 3, 2025 · 5 comments

Comments

@jackingpanda
Copy link

Hello, your project is so useful for my Tiktok HD Downloader project. But can you add more feature like photo url? Like this one?


image

Cause my project could support photo download too. It's so useful if you could add those features. Thank you!

@dinoosauro
Copy link
Owner

Hello! If you need to save those images now you can try opening those photos URL and pasting this script in the command line:

(async () => {
	let [title, id] = [document.querySelector(".css-1tgwhqp-H1PhotoTitle")?.textContent, window.location.href.substring(window.location.href.lastIndexOf("/") + 1)];
	if (id.indexOf("?") !== -1) id = id.substring(0, id.indexOf("?"));
	const images = Array.from(new Set(Array.from(document.querySelectorAll(".swiper-slide.swiper-slide-duplicate > img")).map(item => item.src)));
	const wait = (ms) => new Promise(res => setTimeout(res, ms));
	const getRandom = (max, min) => Math.random() * (max - min) + min
	for (let i = 0; i < images.length; i++) {
		const req = await fetch(images[i]);
		const res = await req.blob();
		Object.assign(document.createElement("a"), {
			href: URL.createObjectURL(res),
			target: "_blank",
			download: `[${i+1}] ${title} [${id}].jpeg`
		}).click();
		await wait(getRandom(50, 400)); // Change here the time the script should wait between downloads (in this case it's a random number between 50 and 400 ms)
	}
})()

Even if I think that you would like to do that programmatically. I could do something with Puppeteer, but developing that in a good way could take some time (and unfortunately I don't have a lot of free time in these days). I'll start writing this script here so that you can use it, and when I'll have done the script in Node.JS for automation I'll publish it

@jasoncuriano
Copy link

jasoncuriano commented Jan 18, 2025

Here's a js script, that uses puppeteer to download tiktok slideshows from the TikTokLinks.txt file. This will save the photos in each slideshow, into a directory named after the video description.

Follow the instructions in the tiktok-to-ytdlp readme to get the output txt file first, then save the following as main.js script on your machine. Once you have node installed, you can then run npm install puppeteer, then put the .txt file in the same directory as the main.js file, and finally run it on the cli node main.js

const puppeteer = require('puppeteer');
const fs = require('fs');
const path = require('path');

(async () => {
    // Read and filter URLs from TikTokLinks.txt
    const urls = fs.readFileSync('TikTokLinks.txt', 'utf8')
        .split('\n')
        .filter(url => url.trim()) // Remove empty lines
        .filter(url => url.includes('/photo/')); // Only keep photo URLs

    console.log(`Found ${urls.length} photo URLs to process`);

    // Launch a headless browser
    const browser = await puppeteer.launch({ headless: false });
    const page = await browser.newPage();

    // Process each URL
    for (const url of urls) {
        try {
            console.log(`Processing ${url}`);
            await page.goto(url, { waitUntil: 'networkidle2' });

            try {
                await page.waitForSelector('.captcha_verify_container', { timeout: 20000 });                
                await page.waitForFunction(
                    () => !document.querySelector('.captcha_verify_container'),
                    { timeout: 300000 }
                );
            } catch (error) {
                console.log('Continuing...');
            }

            // Run the scraping logic with description extraction
            const result = await page.evaluate(() => {
                const username = document.querySelector('h3.css-1n7qc7y-H3AuthorTitle')?.textContent || 
                               window.location.pathname.split('/')[1];
                
                // Get description from meta tag
                const description = document.querySelector('meta[property="og:description"]')?.content || '';
                // Clean up description: remove hashtags and trim
                const cleanDescription = description
                    .split('#')[0]  // Remove hashtags and everything after
                    .trim()
                    .replace(/[\n\r]+/g, ' '); // Replace newlines with spaces
                
                const images = Array.from(new Set(Array.from(document.querySelectorAll(".swiper-slide.swiper-slide-duplicate > img")).map(item => item.src)));
                
                return { username, description: cleanDescription, images };
            });

            // Create directory name with description and username at the end
            const maxLength = 200;
            const usernameSection = ` (@${result.username})`;
            const maxDescLength = maxLength - usernameSection.length;
            
            const isDescriptionTruncated = result.description.length > maxDescLength;
            const truncatedDesc = isDescriptionTruncated
                ? result.description.substring(0, maxDescLength - 3) + '...'
                : result.description;
            
            const dirName = (truncatedDesc + usernameSection).replace(/[<>:"/\\|?*]/g, '_');
            const dirPath = path.join(process.cwd(), dirName);
            
            if (!fs.existsSync(dirPath)) {
                fs.mkdirSync(dirPath, { recursive: true });
            }

            // Save text file with description-username as filename
            const txtFileName = `${result.description} (@${result.username}).txt`.replace(/[<>:"/\\|?*]/g, '_');
            const infoContent = `${result.description}\n@${result.username}\n${url}`;
            fs.writeFileSync(path.join(dirPath, txtFileName), infoContent);

            // Handle downloads
            for (let i = 0; i < result.images.length; i++) {
                const response = await page.goto(result.images[i]);
                const buffer = await response.buffer();
                
                const filename = path.join(dirPath, `[${i+1}].jpeg`);
                fs.writeFileSync(filename, buffer);
                
                await new Promise(res => setTimeout(res, Math.random() * (400 - 50) + 50));
            }

            console.log(`Completed processing ${url}`);
            
            // Optional: Add a delay between processing different URLs
            await new Promise(res => setTimeout(res, 1000));

        } catch (error) {
            console.error(`Error processing ${url}:`, error);
            // Continue with next URL even if current one fails
            continue;
        }
    }

    // Close the browser after all URLs are processed
    await browser.close();
})();

@SushiAwoolf
Copy link

sorry im completely new to this, how do i use this to save slideshows?

@SushiAwoolf
Copy link

I have the txt file, but what do you mean by / how do i 'save the following as main.js script ' ?

@dinoosauro
Copy link
Owner

First of all, @jasoncuriano thanks a lot for the script!

@SushiAwoolf first of all, you need to have Node.js installed on your computer. If not, you can do that from https://nodejs.org/. After you've installed it, copy and paste the code above into a text editor, and then save it in a new folder (you can name it main.js). Now, open the command line in the folder you've saved the file (if you're using Windows, you can do that by writing cmd in the address bar of File Explorer), and write npm install puppeteer. Now, copy the TikTokLinks.txt file into the same folder, and then write on the command line node main.js. A new window will open, and will go to the TikTok webpage. It'll automatically download the images

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants