Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POC of Market Based Pricing #4070

Merged
merged 22 commits into from
Dec 2, 2024
Merged

POC of Market Based Pricing #4070

merged 22 commits into from
Dec 2, 2024

Conversation

d80tb7
Copy link
Collaborator

@d80tb7 d80tb7 commented Nov 29, 2024

Proof Of Concept Of Market Based Pricing. Main changes are as follows:

  • Job submissions can now have an ExperimentalPricingInfo section added. This allows a BidPrice to be set.
  • In the Scheduler configuration pools may be configured to be MarketDriven. If this flag is set then the pool will be use price-based scheduling rahter than fair-share-based scheduling.
  • When using price-based scheduling, jobs are shceduled in price order wioth no consideration to fair share. Jobs are preempted as necessary including self-preemption if you outbid yourself.
  • For MarketDriven pools a SpotPrice metric is produced. This will be zero if the pool is less than 95% utilized, otherswise it will be eqaul to the lowest priced job (running or scheduled).
  • Fair share scheduling is unaffected.

@@ -8,7 +8,7 @@
<env name="ARMADA_EXECUTORAPICONNECTION_FORCENOTLS" value="true" />
<env name="ARMADA_HTTPPORT" value="8085" />
<env name="HOME" value="$USER_HOME$/" />
<env name="KUBECONFIG" value="$PROJECT_DIR$/.kube/internal/config" />
<env name="KUBECONFIG" value="$PROJECT_DIR$/.kube/external/config" />
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intentional. The run config for the executor is incorrect as internal will attempt to connect via the kind network, which won't work from Goland. external will attempt to connect on loclahost which is correct.

@@ -3,6 +3,8 @@ apiVersion: kind.x-k8s.io/v1alpha4
name: armada-test
featureGates:
"KubeletInUserNamespace": true
networking:
apiServerAddress: 0.0.0.0
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intentional. WIthout this the kubeconfig produced by kind will list the address as 0.0.0.0 but a cert that is valid for 127.0.0.1. Settting the apiServerAddress explicitly ensures that kubeconfig alligns with cert.

@d80tb7 d80tb7 changed the title [WIP] POC of Market Based Pricing POC of Market Based Pricing Dec 2, 2024
@d80tb7 d80tb7 enabled auto-merge (squash) December 2, 2024 15:35
@d80tb7 d80tb7 disabled auto-merge December 2, 2024 15:51
@d80tb7 d80tb7 enabled auto-merge (squash) December 2, 2024 15:51
@d80tb7 d80tb7 merged commit d5612f5 into master Dec 2, 2024
28 checks passed
@d80tb7 d80tb7 deleted the f/chrisma/price-poc-2 branch December 2, 2024 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants