Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RDP's in TS OSDK #1035

Open
wants to merge 39 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
246f893
Runtime Derived Property Syntax
nihalbhatnagar Dec 4, 2024
50ca838
Working derive clause
nihalbhatnagar Dec 4, 2024
4e17a4e
Update to use map
nihalbhatnagar Dec 4, 2024
01f5935
Update aggregation keys
nihalbhatnagar Dec 5, 2024
ebe19e4
Update
nihalbhatnagar Dec 5, 2024
4ea27dc
Update PR
nihalbhatnagar Dec 5, 2024
d2d0a0d
Initial test suite
nihalbhatnagar Dec 6, 2024
efbed3d
Working version
nihalbhatnagar Dec 9, 2024
40ee625
Cleanup
nihalbhatnagar Dec 9, 2024
3e535da
Add test case
nihalbhatnagar Dec 9, 2024
20835ea
remove use of crypto
nihalbhatnagar Dec 9, 2024
76a3ee1
Changeset
nihalbhatnagar Dec 9, 2024
73c23fe
E2E testing
nihalbhatnagar Dec 10, 2024
d605787
Osdk.Instance typing update
nihalbhatnagar Dec 10, 2024
214dad7
Rename and cleanup
nihalbhatnagar Dec 10, 2024
73fbffa
Fix UUID
nihalbhatnagar Dec 10, 2024
9070c29
Quick fixes
nihalbhatnagar Dec 13, 2024
ac7f03f
Update aggregations syntax
nihalbhatnagar Dec 16, 2024
2777334
Merge branch 'main' into nihalb/rdp-v1
nihalbhatnagar Dec 16, 2024
a8546ca
Update
nihalbhatnagar Dec 18, 2024
0cd49f5
Merge branch 'main' into nihalb/rdp-v1
nihalbhatnagar Jan 8, 2025
0a89f9c
Update names
nihalbhatnagar Jan 8, 2025
a2f331d
Updated types to produce a shorter definition and also allows us to c…
nihalbhatnagar Jan 15, 2025
bec53ca
Updated tests
nihalbhatnagar Jan 15, 2025
d8fa066
Update tests
nihalbhatnagar Jan 15, 2025
ecb0522
Merge branch 'main' into nihalb/rdp-v1
nihalbhatnagar Jan 15, 2025
abc0b2d
Update to no longer use definitionId
nihalbhatnagar Jan 16, 2025
831f890
Additional type test
nihalbhatnagar Jan 16, 2025
a270994
Rework RDP types (#1113)
ericanderson Jan 27, 2025
7a8847f
Reorganize files and rename Rdp to DerivedProperty
nihalbhatnagar Jan 28, 2025
2659b3b
Osdk.Instance<> for RDP's
nihalbhatnagar Jan 29, 2025
1167fae
Refactor tests
nihalbhatnagar Jan 29, 2025
330dac1
Merge branch 'main' into nihalb/rdp-v1
nihalbhatnagar Jan 29, 2025
eb99bab
Add tests back
nihalbhatnagar Jan 29, 2025
c5686e0
Remove comment
nihalbhatnagar Jan 30, 2025
624b662
Simplify types
ericanderson Jan 30, 2025
2812538
slim the delta more
ericanderson Jan 30, 2025
c2c675f
Update names of builders
nihalbhatnagar Jan 30, 2025
95829f8
Use types instead of interfaces to support better hover
nihalbhatnagar Feb 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/forty-eggs-check.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"@osdk/shared.test": minor
"@osdk/client": minor
"@osdk/api": minor
---

Adds support for runtime derived properties
39 changes: 36 additions & 3 deletions etc/api.report.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,17 @@ export type Augment<X extends ObjectOrInterfaceDefinition, T extends string> = {
// @public (undocumented)
export type Augments = Record<string, string[]>;

// Warning: (ae-forgotten-export) The symbol "FilterableDeriveObjectSet" needs to be exported by the entry point index.d.ts
//
// @public (undocumented)
export interface BaseDeriveObjectSet<Q extends ObjectOrInterfaceDefinition> extends FilterableDeriveObjectSet<Q> {
// Warning: (ae-forgotten-export) The symbol "SingleLinkDeriveObjectSet" needs to be exported by the entry point index.d.ts
// Warning: (ae-forgotten-export) The symbol "ManyLinkDeriveObjectSet" needs to be exported by the entry point index.d.ts
//
// (undocumented)
readonly pivotTo: <L extends LinkNames<Q>>(type: L) => NonNullable<CompileTimeMetadata<Q>["links"][L]["multiplicity"]> extends false ? SingleLinkDeriveObjectSet<LinkedType<Q, L>> : ManyLinkDeriveObjectSet<LinkedType<Q, L>>;
}

// @public (undocumented)
export interface BaseObjectSet<Q extends ObjectOrInterfaceDefinition> {
// (undocumented)
Expand Down Expand Up @@ -338,6 +349,23 @@ export interface DataValueWireToClient {
}[];
}

// @public (undocumented)
export type DeriveClause<Q extends ObjectOrInterfaceDefinition> = {
[key: string]: (baseObjectSet: BaseDeriveObjectSet<Q>) => DerivedPropertyDefinition<ObjectMetadata.Property>;
};

// @public (undocumented)
export type DerivedPropertyDefinition<T extends ObjectMetadata.Property> = {
definitionId: unknown;
type: T;
};

// Warning: (ae-forgotten-export) The symbol "AggregatableDeriveObjectSet" needs to be exported by the entry point index.d.ts
//
// @public (undocumented)
export interface DeriveObjectSet<Q extends ObjectOrInterfaceDefinition> extends BaseDeriveObjectSet<Q>, AggregatableDeriveObjectSet<Q>, SingleLinkDeriveObjectSet<Q> {
}

// @public (undocumented)
export const DistanceUnitMapping: {
centimeter: "CENTIMETERS";
Expand Down Expand Up @@ -589,6 +617,10 @@ export interface ObjectSet<Q extends ObjectOrInterfaceDefinition = any, _UNUSED
};
readonly subtract: (...objectSets: ReadonlyArray<CompileTimeMetadata<Q>["objectSet"]>) => this;
readonly union: (...objectSets: ReadonlyArray<CompileTimeMetadata<Q>["objectSet"]>) => this;
// Warning: (ae-forgotten-export) The symbol "DerivedPropertyExtendedObjectDefinition" needs to be exported by the entry point index.d.ts
//
// (undocumented)
readonly withProperties: <D extends DeriveClause<Q>>(clause: D) => ObjectSet<DerivedPropertyExtendedObjectDefinition<Q, D>>;
}

// @public (undocumented)
Expand Down Expand Up @@ -941,12 +973,13 @@ export type TimeSeriesQuery = {
// @public (undocumented)
export type TwoDimensionalQueryAggregationDefinition = AggregationKeyDataType<"date" | "double" | "timestamp">;

// Warning: (ae-forgotten-export) The symbol "AGG_FOR_TYPE" needs to be exported by the entry point index.d.ts
// Warning: (ae-forgotten-export) The symbol "StringAggregateOption" needs to be exported by the entry point index.d.ts
// Warning: (ae-forgotten-export) The symbol "NumericAggregateOption" needs to be exported by the entry point index.d.ts
// Warning: (ae-forgotten-export) The symbol "PropertyValueClientToWire" needs to be exported by the entry point index.d.ts
//
// @public (undocumented)
export type ValidAggregationKeys<Q extends ObjectOrInterfaceDefinition> = keyof ({
[KK in AggregatableKeys<Q> as `${KK & string}:${AGG_FOR_TYPE<PropertyValueClientToWire[CompileTimeMetadata<Q>["properties"][KK]["type"]]>}`]?: any;
export type ValidAggregationKeys<Q extends ObjectOrInterfaceDefinition, StringOptions extends string = StringAggregateOption, NumericOptions extends string = NumericAggregateOption> = keyof ({
[KK in AggregatableKeys<Q> as `${KK & string}:${number extends PropertyValueClientToWire[CompileTimeMetadata<Q>["properties"][KK]["type"]] ? NumericOptions : string extends PropertyValueClientToWire[CompileTimeMetadata<Q>["properties"][KK]["type"]] ? StringOptions : never}`]?: any;
} & {
$count?: any;
});
Expand Down
11 changes: 8 additions & 3 deletions packages/api/src/aggregate/AggregatableKeys.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,19 @@ type AGG_FOR_TYPE<T> = number extends T ? NumericAggregateOption

export type ValidAggregationKeys<
Q extends ObjectOrInterfaceDefinition,
StringOptions extends string = StringAggregateOption,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to keep all the agg options a bit simpler to parse though, another way to format this type could be keeping the Agg_For_Type and modifying it to take an extra boolean (that controls whether this is for derive or not, default to false) and have it branch to the right agg options that way, rather than having the responsibility on the user to pass in custom options if they want. That being said, if you feel like in the future we'll need to customize agg options even more, this does make it more flexible so up to you

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you do keep it like this, lets remove AGG_FOR_TYPE if its not being used anywhere

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm yeah this does make sense to me, will refactor

NumericOptions extends string = NumericAggregateOption,
> = keyof (
& {
[
KK in AggregatableKeys<Q> as `${KK & string}:${AGG_FOR_TYPE<
KK in AggregatableKeys<Q> as `${KK & string}:${number extends
PropertyValueClientToWire[
CompileTimeMetadata<Q>["properties"][KK]["type"]
]
>}`
] ? NumericOptions
: string extends PropertyValueClientToWire[
CompileTimeMetadata<Q>["properties"][KK]["type"]
] ? StringOptions
: never}`
]?: any;
}
& { $count?: any }
Expand Down
39 changes: 39 additions & 0 deletions packages/api/src/derivedProperties/DeriveClause.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
/*
* Copyright 2024 Palantir Technologies, Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import type {
ObjectOrInterfaceDefinition,
} from "../ontology/ObjectOrInterface.js";
import type {
ObjectMetadata,
PropertyDef,
} from "../ontology/ObjectTypeDefinition.js";
import type { BaseDeriveObjectSet } from "./DeriveObjectSet.js";

export type DerivedPropertyDefinition<
T extends ObjectMetadata.Property,
> = {
definitionId: unknown;
type: T;
};

export type DeriveClause<
Q extends ObjectOrInterfaceDefinition,
> = {
[key: string]: (
baseObjectSet: BaseDeriveObjectSet<Q>,
) => DerivedPropertyDefinition<ObjectMetadata.Property>;
};
117 changes: 117 additions & 0 deletions packages/api/src/derivedProperties/DeriveObjectSet.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
/*
* Copyright 2024 Palantir Technologies, Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import type { ValidAggregationKeys } from "../aggregate/AggregatableKeys.js";
import type { WhereClause } from "../aggregate/WhereClause.js";
import type {
ObjectOrInterfaceDefinition,
PropertyKeys,
} from "../ontology/ObjectOrInterface.js";
import type {
CompileTimeMetadata,
PropertyDef,
} from "../ontology/ObjectTypeDefinition.js";
import type { LinkedType, LinkNames } from "../util/LinkUtils.js";
import type { DerivedPropertyDefinition } from "./DeriveClause.js";

export interface BaseDeriveObjectSet<Q extends ObjectOrInterfaceDefinition>
extends FilterableDeriveObjectSet<Q>
{
readonly pivotTo: <L extends LinkNames<Q>>(
type: L,
) => NonNullable<CompileTimeMetadata<Q>["links"][L]["multiplicity"]> extends
false ? SingleLinkDeriveObjectSet<LinkedType<Q, L>>
: ManyLinkDeriveObjectSet<LinkedType<Q, L>>;
}

export interface DeriveObjectSet<Q extends ObjectOrInterfaceDefinition>
extends
BaseDeriveObjectSet<Q>,
AggregatableDeriveObjectSet<Q>,
SingleLinkDeriveObjectSet<Q>
{}

interface FilterableDeriveObjectSet<
Q extends ObjectOrInterfaceDefinition,
> {
readonly where: (
clause: WhereClause<Q>,
) => this;
}

type CollectAggregations = "collectSet" | "collectList";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parroting gateway names for now. This probably needs a review into valueSet or valueList, but the gateway names should be fine for right now


type BaseDeriveAggregations =
| "approximateDistinct"
| "exactDistinct"
| "approximatePercentile";

export type StringDeriveAggregateOption =
| BaseDeriveAggregations
| CollectAggregations;
export type NumericDeriveAggregateOption =
| "min"
| "max"
| "sum"
| "avg"
| BaseDeriveAggregations
| CollectAggregations;

interface AggregatableDeriveObjectSet<
Q extends ObjectOrInterfaceDefinition,
> extends FilterableDeriveObjectSet<Q> {
readonly aggregate: <
V extends ValidAggregationKeys<
Q,
StringDeriveAggregateOption,
NumericDeriveAggregateOption
>,
>(
aggregationSpecifier: V,
opts?: V extends `${any}:${infer P}`
? P extends CollectAggregations ? { limit: number }
: P extends "approximatePercentile" ? { percentile: number }
: never
: never,
) => DerivedPropertyDefinition<
V extends `${infer N}:${infer P}`
? P extends CollectAggregations ? PropertyDef<
CompileTimeMetadata<Q>["properties"][N]["type"],
"nullable",
"array"
>
: P extends "approximateDistinct" | "exactDistinct" | "$count"
? PropertyDef<"integer">
: PropertyDef<"double">
: never
>;
}

interface SingleLinkDeriveObjectSet<
Q extends ObjectOrInterfaceDefinition,
> extends AggregatableDeriveObjectSet<Q>, BaseDeriveObjectSet<Q> {
readonly selectProperty: <R extends PropertyKeys<Q>>(
propertyName: R,
) => DerivedPropertyDefinition<CompileTimeMetadata<Q>["properties"][R]>;
}

interface ManyLinkDeriveObjectSet<
Q extends ObjectOrInterfaceDefinition,
> extends AggregatableDeriveObjectSet<Q> {
readonly pivotTo: <L extends LinkNames<Q>>(
type: L,
) => ManyLinkDeriveObjectSet<LinkedType<Q, L>>;
}
9 changes: 9 additions & 0 deletions packages/api/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,15 @@ export type {
OsdkObjectLinksObject,
SingleLinkAccessor,
} from "./definitions/LinkDefinitions.js";
export type {
DeriveClause,
DerivedPropertyDefinition,
} from "./derivedProperties/DeriveClause.js";
export type {
BaseDeriveObjectSet,
DeriveObjectSet,
} from "./derivedProperties/DeriveObjectSet.js";

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra new line causes grouping when organizing imports instead of perfect sorting.

export { DurationMapping } from "./groupby/GroupByClause.js";
export type {
AllGroupByValues,
Expand Down
30 changes: 30 additions & 0 deletions packages/api/src/objectSet/ObjectSet.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ import type { AggregateOpts } from "../aggregate/AggregateOpts.js";
import type { AggregateOptsThatErrorsAndDisallowsOrderingWithMultipleGroupBy } from "../aggregate/AggregateOptsThatErrors.js";
import type { AggregationsResults } from "../aggregate/AggregationsResults.js";
import type { WhereClause } from "../aggregate/WhereClause.js";
import type {
DeriveClause,
DerivedPropertyDefinition,
} from "../derivedProperties/DeriveClause.js";
import type {
AsyncIterArgs,
Augments,
Expand All @@ -34,7 +38,9 @@ import type {
} from "../ontology/ObjectOrInterface.js";
import type {
CompileTimeMetadata,
ObjectMetadata,
ObjectTypeDefinition,
PropertyDef,
} from "../ontology/ObjectTypeDefinition.js";
import type { PrimaryKeyType } from "../OsdkBase.js";
import type { ExtractOptions, Osdk } from "../OsdkObjectFrom.js";
Expand Down Expand Up @@ -254,4 +260,28 @@ export interface ObjectSet<
listener: ObjectSetListener<Q, P>,
opts?: ObjectSetListenerOptions<Q, P>,
) => { unsubscribe: () => void };

readonly withProperties: <
D extends DeriveClause<Q>,
>(
clause: D,
) => ObjectSet<
DerivedPropertyExtendedObjectDefinition<
Q,
D
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm okay with it for now but this feels like a place where having the simpler types that are not based on functions produces more readable types? We have to do the extraction work anyway to create things like properties and props below but we are required to keep the function hierarchy in the type generics as this object is passed around instead of resolving them at the time withProperties is called and thus carrying a lot less. Like we could just be carrying { foo: number, bar: string } lets say. Or even { foo: PropertyDefinition, bar: PropertyDefinition }.

>
>;
}

type DerivedPropertyExtendedObjectDefinition<
K extends ObjectOrInterfaceDefinition,
D extends DeriveClause<K>,
> = {
__DefinitionMetadata: {
properties: {
nihalbhatnagar marked this conversation as resolved.
Show resolved Hide resolved
[T in Extract<keyof D, string>]: D[T] extends
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd have to check but my gut is keyof D & string is better for the compiler than the conditional type that Extract causes.

(baseObjectSet: any) => DerivedPropertyDefinition<infer P> ? P
: never;
};
};
} & K;
Loading
Loading