Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convenient Pagination API, Browser Support, Missing API fields #60

Open
wants to merge 38 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
9396cd3
working all
pyrogenic May 21, 2021
bb7817b
cleanup
pyrogenic May 21, 2021
ec550c8
add custom fields
pyrogenic May 23, 2021
37fb0ff
option to override fetch implementation
pyrogenic May 24, 2021
7258359
formatting
pyrogenic May 24, 2021
e2fc3dc
request cache
pyrogenic May 26, 2021
2a3b68f
avatar_url
pyrogenic May 26, 2021
ddaae5e
modernize build, add build-dev script
pyrogenic May 26, 2021
2b48c95
can't use query parameterts with POST
pyrogenic May 27, 2021
247cd25
lint
pyrogenic May 27, 2021
35e5636
fix typing of inventory result
pyrogenic Jul 6, 2021
8f76c28
add format details
pyrogenic Jul 10, 2021
a63f437
missing name field
pyrogenic Jul 18, 2021
8f65975
browser-save headers
pyrogenic Jul 20, 2021
3f59c72
fix another unsafe browser header
pyrogenic Jul 25, 2021
16fe299
lint
pyrogenic Jul 25, 2021
3d6cb6c
add master notes
pyrogenic Aug 1, 2021
871b35e
track artists!
pyrogenic Aug 7, 2021
4a24c56
fix fields in listing
pyrogenic Aug 25, 2021
25d57a7
fix createListing URL
pyrogenic Sep 20, 2021
7741df9
lint
pyrogenic Sep 20, 2021
e971854
enable breakpointing
pyrogenic Nov 27, 2021
08db146
typescript 4.5
pyrogenic May 9, 2022
4d5ace0
Discogs 443s if there are multiple content types.
pyrogenic Oct 8, 2023
e70dc85
Lost in last commit
pyrogenic Oct 8, 2023
958f48c
Merge pull request #1 from aknorw/master
pyrogenic Feb 10, 2024
a2aa543
partial merge
pyrogenic Feb 10, 2024
26cd6a6
missing file from merge
pyrogenic Feb 10, 2024
c10f132
complete fetch merge
pyrogenic Feb 10, 2024
88e694d
finish merge of discojs
pyrogenic Feb 10, 2024
427c50a
Windows build support
pyrogenic Feb 11, 2024
277a9f2
do not add API prefix to REST API links, since they are already full …
pyrogenic Feb 11, 2024
37f9633
prettier
pyrogenic Feb 11, 2024
4eecc6d
add `allowUnsafeHeaders` (defaults to true) for use in browsers (with…
pyrogenic Feb 11, 2024
2ef0508
undo old removal of auto-add of prettified source
pyrogenic Feb 11, 2024
3f7dc7a
use `body` rather than `query` for `editCustomFieldForInstance`
pyrogenic Apr 1, 2024
b359ebc
prevent multiple 'content-type' headers (see https://github.com/tiang…
pyrogenic Apr 1, 2024
51eca9b
restore header setting
pyrogenic Apr 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions models/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,29 @@ import { ExportItemIO } from './inventory'

export type EmptyResponse = {}

export const ErrorResponseIO = t.intersection([
t.type({
message: t.string,
}),
t.partial({
detail: t.array(
t.type({
loc: t.array(t.string),
msg: t.string,
type: t.string,
}),
),
}),
])

export type ErrorResponse = t.TypeOf<typeof ErrorResponseIO>

export type Pagination = t.TypeOf<typeof PaginationIO>
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have a Pagination type, named this one PaginationResponse.


export interface IPaginated {
pagination: Pagination
}

export type IdentityResponse = t.TypeOf<typeof IdentityIO>

export type UserProfileResponse = t.TypeOf<typeof UserIO>
Expand Down Expand Up @@ -57,6 +80,13 @@ export const FoldersResponseIO = t.type({
})
export type FoldersResponse = t.TypeOf<typeof FoldersResponseIO>

export const FieldIO = t.type({
field_id: t.Integer,
value: t.string,
})

export type Field = t.TypeOf<typeof FieldIO>

/**
* @internal
*/
Expand All @@ -69,6 +99,8 @@ export const FolderReleasesResponseIO = t.type({
rating: t.Integer,
date_added: t.string,
basic_information: ReleaseMinimalInfoIO,
folder_id: t.Integer,
notes: t.array(FieldIO),
}),
),
})
Expand Down
1 change: 1 addition & 0 deletions models/artist.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ const ArtistBaseIO = t.intersection([
ResourceURLIO,
t.type({
id: t.Integer,
name: t.string,
profile: t.string,
data_quality: makeEnumIOType(DataQualityEnum),
releases_url: t.string,
Expand Down
15 changes: 13 additions & 2 deletions models/marketplace.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { makeEnumIOType } from './helpers'
import {
CurrenciesEnum,
EditOrderStatusesEnum,
ListingStatusesEnum,
InventoryStatusesEnum,
OrderMessageTypesEnum,
ReleaseConditionsEnum,
SleeveConditionsEnum,
Expand Down Expand Up @@ -70,7 +70,7 @@ export const ListingIO = t.intersection([
ResourceURLIO,
t.type({
id: t.Integer,
status: makeEnumIOType(ListingStatusesEnum),
status: makeEnumIOType(InventoryStatusesEnum),
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I chose not to introduce this change yet, as this is only the case when users are authenticated. IMHO we should have 2 different codecs/types depending on whether the user is authenticated.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I didn't know that difference, I only saw it not match. Is it the user being authenticated at all, or the authenticated user being the owner of the listing?

I'll check out the code paths I encounter this and see what might make sense.

release: t.intersection([
ResourceURLIO,
t.type({
Expand Down Expand Up @@ -113,9 +113,20 @@ export const ListingIO = t.intersection([
audio: t.boolean,
uri: t.string,
}),
// If the user is authorized, the listing will contain a in_cart boolean field indicating whether or not this listing is in their cart.
t.partial({
in_cart: t.boolean,
}),
// If the authorized user is the listing owner the listing will include the weight, format_quantity, external_id, and location keys.
t.intersection([
t.type({
weight: t.number,
format_quantity: t.number,
external_id: t.string,
location: t.string,
}),
t.type({}),
]),
])

/**
Expand Down
3 changes: 3 additions & 0 deletions models/master.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ export const MasterIO = t.intersection([
videos: t.array(VideoIO),
uri: t.string,
}),
t.partial({
notes: t.string,
}),
])

export type Master = t.TypeOf<typeof MasterIO>
Expand Down
27 changes: 20 additions & 7 deletions models/release.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,15 @@ export const ReleaseArtistIO = t.intersection([
/**
* @internal
*/
const ReleaseFormatIO = t.type({
const ReleaseFormatIOConcrete = t.type({
name: t.string,
qty: t.string,
})
const ReleaseFormatIOOptional = t.partial({
text: t.string,
descriptions: t.array(t.string),
})
const ReleaseFormatIO = t.intersection([ReleaseFormatIOConcrete, ReleaseFormatIOOptional])

/**
* @internal
Expand All @@ -67,6 +72,9 @@ const ReleaseEntityIO = t.intersection([
entity_type_name: t.string,
catno: t.string,
}),
t.partial({
thumbnail_url: t.string,
}),
])

/**
Expand All @@ -80,12 +88,17 @@ const ReleaseIdentifierIO = t.type({
/**
* @internal
*/
export const TrackIO = t.type({
type_: t.string,
title: t.string,
position: t.string,
duration: t.string,
})
export const TrackIO = t.intersection([
t.type({
type_: t.string,
title: t.string,
position: t.string,
duration: t.string,
}),
t.partial({
artists: t.array(ReleaseArtistIO),
}),
])

/**
* @internal
Expand Down
1 change: 1 addition & 0 deletions models/user.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ export const UserIO = t.intersection([
username: t.string,
name: t.string,
profile: t.string,
avatar_url: t.string,
home_page: t.string,
location: t.string,
registered: t.string,
Expand Down
5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "discojs",
"version": "2.2.0",
"version": "2.3.0",
"description": "Easiest way to use the Discogs API in Javascript/Typescript",
"author": "aknorw",
"license": "MIT",
Expand All @@ -17,14 +17,15 @@
"types": "./lib/index.d.ts",
"scripts": {
"clean": "rimraf lib",
"pretty": "prettier --config .prettierrc --write '{models,src}/**/*.{ts,json}'",
"pretty": "prettier --config .prettierrc --write \"{models,src}/**/*.{ts,json}\"",
"lint": "eslint --ext .ts,.json models/ src/",
"test:spec": "cross-env NODE_ENV=test jest --testMatch='**/*.spec.ts'",
"test:e2e": "cross-env NODE_ENV=test jest --testMatch='**/*.e2e.ts' --runInBand",
"test": "yarn build && yarn test:spec && yarn test:e2e",
"docs": "typedoc --inputFiles src/index.ts --exclude \"**/*+(.spec).ts\" --excludeNotExported --excludePrivate --stripInternal",
"prebuild": "yarn run clean",
"build": "cross-env NODE_ENV=production rollup -c",
"build-dev": "cross-env NODE_ENV=development rollup -c",
"prepare": "husky install"
},
"repository": {
Expand Down
48 changes: 48 additions & 0 deletions src/allNext.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
import { IPaginated } from '../models/api'
import type { Discojs } from './discojs'

export class AllNext {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I felt this implementation was a bit hard to follow, as you first have to call a method, then call client.all.
Instead of doing that, I introduced dedicated all methods that returns async iterators:

  • getAllMasterVersions
  • getAllArtistReleases
  • getAllLabelReleases
  • getAllRecentExports
  • getAllInventoryForUser
  • getAllInventory
  • listAllOrders
  • listAllItemsByReleaseForUser
  • listAllItemsByRelease
  • listAllItemsInFolderForUser
  • listAllItemsInFolder
  • getAllSubmissionsForUser
  • getAllSubmissions
  • getAllContributionsForUser
  • getAllContributions
  • getAllListsForUser
  • getAllLists
  • getAllWantlistForUser
  • getAllWantlist

The example you have in the PR description would become:

async function updateCollection() {
  for await (const { releases } of client.listAllItemsInFolder()) {
    for (const release of releases) {
      await addToCollection(release)
    }
  }
}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems much more user-friendly, if less magical. 😉

I probably chose my method based on writing the least code and exercising some TypeScript-fu.

/**
* @internal
*
* Gets the next page of a paginated result
*/
next<TResponse extends IPaginated>(this: Discojs, response: TResponse) {
const { next } = response.pagination.urls
if (next === undefined) {
return Promise.resolve(undefined)
}
return this.fetcher.schedule<TResponse>(next)
}

/**
* Retrieve all resources from a paginated endpoint.
* @param this
* @param key the name of the field in `TResponse` you wish to aggregate
* @param response the starting page for the paginated response
* @param onProgress optional callback to invoke as each page is retrieved
* @typeparam TKey the name of the field in `TResponse` you wish to aggregate (inferred from `key`)
* @typeparam TResultElement the type of record (inferred from `key` and `response`)
* @typeparam TResponse the type of the response (inferred from `response`)
* @returns an array containing all elements from the different pages concatenated together
*/
async all<TKey extends string, TResultElement, TResponse extends IPaginated & { [K in TKey]: TResultElement[] }>(
this: Discojs,
key: TKey,
response: TResponse | undefined,
onProgress?: (data: TResultElement[]) => void,
) {
let result: TResultElement[] = []
while (response !== undefined) {
const data = response[key]
onProgress?.(data)
result = result.concat(data)
// eslint-disable-next-line no-await-in-loop, no-param-reassign
response = await this.next(response)
if (response === undefined) {
break
}
}
return result
}
}
3 changes: 3 additions & 0 deletions src/discojs.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { CurrenciesEnum, ReleaseConditionsEnum, SleeveConditionsEnum } from './enums'
import { applyMixins, FetcherOptions, Fetcher, OutputFormat } from './utils'

import { AllNext } from './allNext'
import { Database } from './database'
import { MarketPlace } from './marketplace'
import { InventoryExport } from './inventoryExport'
Expand All @@ -19,6 +20,7 @@ export interface Discojs
UserLists,
Database,
MarketPlace,
AllNext,
InventoryExport,
InventoryUpload {}

Expand Down Expand Up @@ -88,6 +90,7 @@ applyMixins(Discojs, [
UserLists,
Database,
MarketPlace,
AllNext,
InventoryExport,
InventoryUpload,
])
3 changes: 2 additions & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
export * from './discojs'
export * from './enums'
export * from './errors'
export { SortOrdersEnum } from './utils'
export { ResultCache, SortOrdersEnum } from './utils'
export { IPaginated, Pagination } from '../models/api'
3 changes: 2 additions & 1 deletion src/userCollection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -332,8 +332,9 @@ export class UserCollection {
const username = await this.getUsername()
return this.fetcher.schedule<EmptyResponse>(
`/users/${username}/collection/folders/${folderId}/releases/${releaseId}/instances/${instanceId}/fields/${fieldId}`,
{ value },
undefined,
HTTPVerbsEnum.POST,
{ value },
)
}

Expand Down
Loading