Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Much more efficient way to query user questions found #186

Open
JakeIwen opened this issue Mar 4, 2019 · 1 comment
Open

Much more efficient way to query user questions found #186

JakeIwen opened this issue Mar 4, 2019 · 1 comment

Comments

@JakeIwen
Copy link

JakeIwen commented Mar 4, 2019

No more HTML requests/scraping!

//return ALL available questions for a given user
//window.ACCESS_TOKEN required for API call

_OKCP.getApiAnswers = async function(userId){
  
  const answers = [];
  const params = { api: 1, type: "GET" };
  let cursor = '';
  let end = false;
  
  do {    //must be async loop to get cursor from each response
    
    const path = `/profile/${userId}/answers?after=${cursor}`;
    const {data, paging} = await window.OkC.api(path, params);

    if(!data || !paging) {
      return answers;   //bad response: bail. This is very rare. 
    }
    
    answers.push(...data);
    cursor = paging.cursors.after;
    end = paging.end;
    
  } while(!end)
  
  return answers;
  
}

This is presumably similar to whats happening when you scroll down on the new questions page and it loads more answers.

I have a monkeypatch method of incorporating this on my fork that sort of makes your parser function think its still reading HTML. Not something I think that should be used in production though.

window.OkC is from the actual okcapi.js, attained from okcupid.com.
https://github.com/JakeIwen/chrome-okc-plugin/blob/master/plugin/okcapi.js

Sorry fo lack of pull request, but I fear anything I make could have unforeseen (on my part) consequences / errors.

@phoebejaffe
Copy link
Owner

Oh excellent! This seems like a good thing to integrate.

The latest version doesn't do any HTML scraping (as of January I think), but this function might make things even easier.

Might be a while before I get around to it though since I've got a lot going on right now. Hopefully soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants