Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make individual colors from Vega color schemes accessible from Altair #2617

Open
1 task
Tracked by #2861
joelostblom opened this issue Jun 15, 2022 · 8 comments
Open
1 task
Tracked by #2861
Labels
enhancement vega: vega Requires upstream action in `vega`

Comments

@joelostblom
Copy link
Contributor

joelostblom commented Jun 15, 2022

Tasks

Description

I think it would be valuable to make all the individual colors from Vega schemes accessible programmatically from Altair. Since Vega stores them in a single file, we could parse that and create a Python dictionary that could be used to access single colors from each color scheme. Something like this:

Code block

pd.read_table(
    'https://raw.githubusercontent.com/vega/vega/v5.21.0/packages/vega-scale/src/palettes.js',
    skipinitialspace=True,
    sep=':',
).drop(
    index=['export const discrete = {', '};']
).iloc[:, 0].str.replace(
    "'", ""
).str.replace(
    ',', ''
).apply(
    lambda x: ["#" + x[i:i+6] for i in range(0, len(x), 6)]
).to_dict()
{'blues': [
  '#cfe1f2',
  '#bed8ec',
  '#a8cee5',
  '#8fc1de',
  '#74b2d7',
  '#5ba3cf',
  '#4592c6',
  '#3181bd',
  '#206fb2',
  '#125ca4',
  '#0a4a90'],
 'greens': [
  '#d3eecd',
  '#c0e6ba',
  '#abdda5',
  '#94d391',
  '#7bc77d',
  '#60ba6c',
  '#46ab5e',
  '#329a51',
  '#208943',
  '#0e7735',
  '#036429']
...

@dangotbanned
Copy link
Member

dangotbanned commented Jan 3, 2025

@joelostblom I'm adding some context for what is going on in your script, and linking docs for Vega/Color Schemes

The lambda appears to be similar to d3-scale-chromatic/src/colors.js, with an example of usage in category10.js.
In short, splitting a string containing multiple hexadecimal colors

src/categorical/category10.js

https://github.com/d3/d3-scale-chromatic/blob/2c52792197299346b7bdb94322bb4dff8f554fea/src/categorical/category10.js#L3

A related comment by @domoritz (vega/vega#3843 (comment))

import colors from "../colors.js";

export default colors("1f77b4ff7f0e2ca02cd627289467bd8c564be377c27f7f7fbcbd2217becf");

The current palettes.js has changed since this issue was opened.
Lines L1-L12 and L78-L94 would probably break the initial parser idea.

I do agree these would be nice to have available in altair.
I think we should try to push for a Public API for this upstream in (https://github.com/vega/vega) - since that would be safer than dependending on internal library code.
That route was really helpful for me when I started reaching for vl_convert internals in vega/vl-convert#191

@dangotbanned dangotbanned added the vega: vega Requires upstream action in `vega` label Jan 7, 2025
@joelostblom
Copy link
Contributor Author

I think we should try to push for a Public API for this upstream in (https://github.com/vega/vega) - since that would be safer than dependending on internal library code.

Yes, I agree that this is the better option

@dangotbanned
Copy link
Member

@joelostblom something that might help push this forward is an example that would benefit from more granular access to the color palettes.

It could be an existing altair example, one from an upstream gallery that is difficult to reproduce or something entirely new.

Maybe even adapting some of the code from @mattijn's work in doc/user_guide/color_usage.rst.
I imagine being able to simplify our docs with help from upstream would be a net positive - and hopefully a step towards merging #3021

@dangotbanned
Copy link
Member

cc @hydrosquall regarding #2617 (comment)

Wondering if you had any thoughts how we could expose the colors within a scheme from vega?

I feel like this would be simpler than vega/vega#1938

@hydrosquall
Copy link
Member

I think vega/vega#1938 might not help with this issue, since Vega expressions do not have knowledge of color palettes.

A naive Altair question as I haven't changed it before - do you have the ability to Altair's Python code ask Vega JS runtime for a response, so Vega can be an input to Python rather than just an output? If you can't do it at runtime, you could write a json file into Altair in a CI step.

The schemes are queryable through the same API used to add new schemes, so you wouldn't have to do parsing of a JS file from a URL (though that approach is very ingenious).

https://vega.github.io/vega/docs/api/extensibility/#schemes

@dangotbanned
Copy link
Member

Clarification

I think vega/vega#1938 might not help with this issue, since Vega expressions do not have knowledge of color palettes.

Apologies @hydrosquall, the link I was drawing between these issues wasn't very clear (my bad).

What they have in common is that resolving either could result in new .json describing some currently underspecified functionality.

So if that was the goal here, I think we'd be able to get there with less effort than (vega/vega#1938).
As you've mentioned in (#2617 (comment)), we could evaluate this within the existing Vega JS runtime.

Re altair

A naive Altair question as I haven't changed it before - do you have the ability to Altair's Python code ask Vega JS runtime for a response, so Vega can be an input to Python rather than just an output?
If you can't do it at runtime, you could write a json file into Altair in a CI step.

I don't think we can interact with the Vega JS runtime natively (related #3365).
It is possible with additional (pretty heavy) dependencies (e.g. #3630, docs) - but I'd prefer pre-evaluating here so the colors are available for all users.

Right now, we have the schemes by name only:

altair.vegalite.v5.schema._typing.ColorScheme_T

ColorScheme_T: TypeAlias = Literal[
"accent",
"category10",
"category20",
"category20b",
"category20c",
"dark2",
"paired",
"pastel1",
"pastel2",
"set1",
"set2",
"set3",
"tableau10",
"tableau20",
"observable10",
"blues",
"tealblues",
"teals",
"greens",
"browns",
"greys",
"purples",
"warmgreys",
"reds",
"oranges",
"turbo",
"viridis",
"inferno",
"magma",
"plasma",
"cividis",
"bluegreen",
"bluegreen-3",
"bluegreen-4",
"bluegreen-5",
"bluegreen-6",
"bluegreen-7",
"bluegreen-8",
"bluegreen-9",
"bluepurple",
"bluepurple-3",
"bluepurple-4",
"bluepurple-5",
"bluepurple-6",
"bluepurple-7",
"bluepurple-8",
"bluepurple-9",
"goldgreen",
"goldgreen-3",
"goldgreen-4",
"goldgreen-5",
"goldgreen-6",
"goldgreen-7",
"goldgreen-8",
"goldgreen-9",
"goldorange",
"goldorange-3",
"goldorange-4",
"goldorange-5",
"goldorange-6",
"goldorange-7",
"goldorange-8",
"goldorange-9",
"goldred",
"goldred-3",
"goldred-4",
"goldred-5",
"goldred-6",
"goldred-7",
"goldred-8",
"goldred-9",
"greenblue",
"greenblue-3",
"greenblue-4",
"greenblue-5",
"greenblue-6",
"greenblue-7",
"greenblue-8",
"greenblue-9",
"orangered",
"orangered-3",
"orangered-4",
"orangered-5",
"orangered-6",
"orangered-7",
"orangered-8",
"orangered-9",
"purplebluegreen",
"purplebluegreen-3",
"purplebluegreen-4",
"purplebluegreen-5",
"purplebluegreen-6",
"purplebluegreen-7",
"purplebluegreen-8",
"purplebluegreen-9",
"purpleblue",
"purpleblue-3",
"purpleblue-4",
"purpleblue-5",
"purpleblue-6",
"purpleblue-7",
"purpleblue-8",
"purpleblue-9",
"purplered",
"purplered-3",
"purplered-4",
"purplered-5",
"purplered-6",
"purplered-7",
"purplered-8",
"purplered-9",
"redpurple",
"redpurple-3",
"redpurple-4",
"redpurple-5",
"redpurple-6",
"redpurple-7",
"redpurple-8",
"redpurple-9",
"yellowgreenblue",
"yellowgreenblue-3",
"yellowgreenblue-4",
"yellowgreenblue-5",
"yellowgreenblue-6",
"yellowgreenblue-7",
"yellowgreenblue-8",
"yellowgreenblue-9",
"yellowgreen",
"yellowgreen-3",
"yellowgreen-4",
"yellowgreen-5",
"yellowgreen-6",
"yellowgreen-7",
"yellowgreen-8",
"yellowgreen-9",
"yelloworangebrown",
"yelloworangebrown-3",
"yelloworangebrown-4",
"yelloworangebrown-5",
"yelloworangebrown-6",
"yelloworangebrown-7",
"yelloworangebrown-8",
"yelloworangebrown-9",
"yelloworangered",
"yelloworangered-3",
"yelloworangered-4",
"yelloworangered-5",
"yelloworangered-6",
"yelloworangered-7",
"yelloworangered-8",
"yelloworangered-9",
"darkblue",
"darkblue-3",
"darkblue-4",
"darkblue-5",
"darkblue-6",
"darkblue-7",
"darkblue-8",
"darkblue-9",
"darkgold",
"darkgold-3",
"darkgold-4",
"darkgold-5",
"darkgold-6",
"darkgold-7",
"darkgold-8",
"darkgold-9",
"darkgreen",
"darkgreen-3",
"darkgreen-4",
"darkgreen-5",
"darkgreen-6",
"darkgreen-7",
"darkgreen-8",
"darkgreen-9",
"darkmulti",
"darkmulti-3",
"darkmulti-4",
"darkmulti-5",
"darkmulti-6",
"darkmulti-7",
"darkmulti-8",
"darkmulti-9",
"darkred",
"darkred-3",
"darkred-4",
"darkred-5",
"darkred-6",
"darkred-7",
"darkred-8",
"darkred-9",
"lightgreyred",
"lightgreyred-3",
"lightgreyred-4",
"lightgreyred-5",
"lightgreyred-6",
"lightgreyred-7",
"lightgreyred-8",
"lightgreyred-9",
"lightgreyteal",
"lightgreyteal-3",
"lightgreyteal-4",
"lightgreyteal-5",
"lightgreyteal-6",
"lightgreyteal-7",
"lightgreyteal-8",
"lightgreyteal-9",
"lightmulti",
"lightmulti-3",
"lightmulti-4",
"lightmulti-5",
"lightmulti-6",
"lightmulti-7",
"lightmulti-8",
"lightmulti-9",
"lightorange",
"lightorange-3",
"lightorange-4",
"lightorange-5",
"lightorange-6",
"lightorange-7",
"lightorange-8",
"lightorange-9",
"lighttealblue",
"lighttealblue-3",
"lighttealblue-4",
"lighttealblue-5",
"lighttealblue-6",
"lighttealblue-7",
"lighttealblue-8",
"lighttealblue-9",
"blueorange",
"blueorange-3",
"blueorange-4",
"blueorange-5",
"blueorange-6",
"blueorange-7",
"blueorange-8",
"blueorange-9",
"blueorange-10",
"blueorange-11",
"brownbluegreen",
"brownbluegreen-3",
"brownbluegreen-4",
"brownbluegreen-5",
"brownbluegreen-6",
"brownbluegreen-7",
"brownbluegreen-8",
"brownbluegreen-9",
"brownbluegreen-10",
"brownbluegreen-11",
"purplegreen",
"purplegreen-3",
"purplegreen-4",
"purplegreen-5",
"purplegreen-6",
"purplegreen-7",
"purplegreen-8",
"purplegreen-9",
"purplegreen-10",
"purplegreen-11",
"pinkyellowgreen",
"pinkyellowgreen-3",
"pinkyellowgreen-4",
"pinkyellowgreen-5",
"pinkyellowgreen-6",
"pinkyellowgreen-7",
"pinkyellowgreen-8",
"pinkyellowgreen-9",
"pinkyellowgreen-10",
"pinkyellowgreen-11",
"purpleorange",
"purpleorange-3",
"purpleorange-4",
"purpleorange-5",
"purpleorange-6",
"purpleorange-7",
"purpleorange-8",
"purpleorange-9",
"purpleorange-10",
"purpleorange-11",
"redblue",
"redblue-3",
"redblue-4",
"redblue-5",
"redblue-6",
"redblue-7",
"redblue-8",
"redblue-9",
"redblue-10",
"redblue-11",
"redgrey",
"redgrey-3",
"redgrey-4",
"redgrey-5",
"redgrey-6",
"redgrey-7",
"redgrey-8",
"redgrey-9",
"redgrey-10",
"redgrey-11",
"redyellowblue",
"redyellowblue-3",
"redyellowblue-4",
"redyellowblue-5",
"redyellowblue-6",
"redyellowblue-7",
"redyellowblue-8",
"redyellowblue-9",
"redyellowblue-10",
"redyellowblue-11",
"redyellowgreen",
"redyellowgreen-3",
"redyellowgreen-4",
"redyellowgreen-5",
"redyellowgreen-6",
"redyellowgreen-7",
"redyellowgreen-8",
"redyellowgreen-9",
"redyellowgreen-10",
"redyellowgreen-11",
"spectral",
"spectral-3",
"spectral-4",
"spectral-5",
"spectral-6",
"spectral-7",
"spectral-8",
"spectral-9",
"spectral-10",
"spectral-11",
"rainbow",
"sinebow",
]

This is still helpful, but say for example "bluegreen" we can only know through trial-and-error what was encoded at each stop:

"bluegreen",
"bluegreen-3",
"bluegreen-4",
"bluegreen-5",
"bluegreen-6",
"bluegreen-7",
"bluegreen-8",
"bluegreen-9",

vega != vega-lite

I hadn't noticed before writing this, but we have 300+ color scheme names and that is a huge jump from https://vega.github.io/vega/docs/schemes/
Leads me to think the logic for this would need to be based on vega-lite, rather than vega.

Haven't quite worked out where this takes place - maybe related to vega-lite/src/scale.ts?

@hydrosquall
Copy link
Member

hydrosquall commented Jan 20, 2025

Hi @dangotbanned , thanks very much for the clarifications and helpful links around expr evaluation and theme testing.

I was thinking that when altair runs in jupyter it could communicate with Vega via the Jupyter widgets API (as we did in this streamlit POC ), but since altair can run without jupyter, this isn't a global solution.

I think I see the connection you're drawing between the projects now - if there was a structured (JSON) representation of some vega JS capability, be it the colors (and sampling strategy when mixed stop counts are used) per palette, or the list of all expr, it keeps python code up to date without reimplementing each.


Stepping from the code I'm curious about the end goal. If users are able to access the color definitions in python, what would they use it for? Would they be passed directly back into an altair call, or used in other scripts?

One alternative devtool if they're not planning to use the colors right away would be to offer this info via a devtool page where you can

  • Browse all vega palette names (including bluegreen-n) varieties via selector
  • Type a palette name, get back the color codes

Alternately, we could augment the docs so clicking on a color block copies the hex code for that color to clipboard or displays in a tooltip, similar to how other color / design system sites work (e.g. material)

@hydrosquall
Copy link
Member

I hadn't noticed before writing this, but we have 300+ color scheme names and that is a huge jump from https://vega.github.io/vega/docs/schemes/

Ah good find, it looks like those color scheme names originates in vega. Upon inspection the the Vega docs list discrete (rather than interpolator function)-${n} variants of each palette behind a collapsible button, but only list the suffix rather than the full name:

https://github.com/vega/vega/blob/8fc129a6f8a11e96449c4ac0f63de0e5bfc7254c/packages/vega-typings/types/spec/scheme.d.ts#L30-L44

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement vega: vega Requires upstream action in `vega`
Projects
None yet
Development

No branches or pull requests

3 participants