Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

issue of reading visiumHD file #10

Open
Sophieliula opened this issue Jul 8, 2024 · 7 comments
Open

issue of reading visiumHD file #10

Sophieliula opened this issue Jul 8, 2024 · 7 comments

Comments

@Sophieliula
Copy link

hd = cs.read_visiumHD(hd_dir)

  1. loading 2um binned data...
    Formatting cell polygons...
    Progress: 100%[==================================================] Time: 0:00:5039m
  2. 2um binned data loaded!
  3. loading 8um binned data...
    Formatting cell polygons...
    Progress: 100%[==================================================] Time: 0:00:02
  4. 8um binned data loaded!
  5. loading 16um binned data...
    Formatting cell polygons...
  6. 16um binned data loaded!
    MethodError: no method matching Union{Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGBA{FixedPointNumbers.N0f8}}}(::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}})

Stacktrace:
[1] convert(#unused#::Type{Union{Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGBA{FixedPointNumbers.N0f8}}}}, a::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}})
@ Base ./array.jl:554
[2] convert(#unused#::Type{Union{Nothing, Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGBA{FixedPointNumbers.N0f8}}}}, x::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}})
@ Base ./some.jl:36
[3] CellScopes.VisiumImgObject(highresImage::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}}, lowresImage::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}}, fullresImage::Nothing, detectedTissue::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, alignedImage::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, jsonParameters::Dict{String, Any})
@ CellScopes ~/.julia/packages/CellScopes/pM38e/src/spatial/sp_objects.jl:249
[4] read_visiumHD(hd_dir::String; min_genes::Vector{Int64}, min_cells::Vector{Int64}, prefix::Nothing, postfix::Nothing, default_bin::String)
@ CellScopes ~/.julia/packages/CellScopes/pM38e/src/fileio.jl:660
[5] read_visiumHD(hd_dir::String)
@ CellScopes ~/.julia/packages/CellScopes/pM38e/src/fileio.jl:628
[6] top-level scope
@ In[6]:1
@HaojiaWu
Copy link
Owner

HaojiaWu commented Jul 9, 2024

I tried using hd = cs.read_visiumHD(hd_dir) to read the Spaceranger output from two samples again, and I couldn't reproduce the error you came across. Please make sure that hd_dir is the output directory from Spaceranger to use this function correctly. Other troubleshooting steps you can take include reinstalling CellScopes using Julia 1.8.4. I tested CellScopes on both Julia 1.7.3 and Julia 1.8.4, and it works fine. Other users have reported dependency issues with newer versions of Julia.

@liangxs7015
Copy link

Same issue on both Julia 1.8.4 and 1.7.3, any help for this?

  1. loading 2um binned data...
    Formatting cell polygons...
    Progress: 100%[==================================================] Time: 0:00:4239m
  2. 2um binned data loaded!
  3. loading 8um binned data...
    Formatting cell polygons...
    Progress: 100%[==================================================] Time: 0:00:05
  4. 8um binned data loaded!
  5. loading 16um binned data...
    Formatting cell polygons...
    Progress: 100%[==================================================] Time: 0:00:02
  6. 16um binned data loaded!
    MethodError: no method matching Union{Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGBA{FixedPointNumbers.N0f8}}}(::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}})

Stacktrace:
[1] convert(#unused#::Type{Union{Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGBA{FixedPointNumbers.N0f8}}}}, a::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}})
@ Base ./array.jl:554
[2] convert(#unused#::Type{Union{Nothing, Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGBA{FixedPointNumbers.N0f8}}}}, x::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}})
@ Base ./some.jl:36
[3] CellScopes.VisiumImgObject(highresImage::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}}, lowresImage::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}}, fullresImage::Nothing, detectedTissue::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, alignedImage::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, jsonParameters::Dict{String, Any})
@ CellScopes ~/.julia/packages/CellScopes/XZUoC/src/spatial/sp_objects.jl:249
[4] read_visiumHD(hd_dir::String; min_genes::Vector{Int64}, min_cells::Vector{Int64}, prefix::Nothing, postfix::Nothing, default_bin::String)
@ CellScopes ~/.julia/packages/CellScopes/XZUoC/src/fileio.jl:660
[5] read_visiumHD(hd_dir::String)
@ CellScopes ~/.julia/packages/CellScopes/XZUoC/src/fileio.jl:628
[6] top-level scope
@ In[2]:1

@HaojiaWu
Copy link
Owner

I am transitioning everything to Julia 1.10.4, as older versions have too many dependency issues. Could you please try this and let me know if you come across any problems? Thanks.
https://github.com/HaojiaWu/CellScopes.jl/tree/dev

@liangxs7015
Copy link

Thanks for your reply, when running 'Pkg.add(url="https://github.com/TheHumphreysLab/CellScopes.jl")', I saw some issues:
┌ Interpolations
│ WARNING: method definition for checkbounds at /home/liangxs/.julia/packages/Interpolations/y4lLj/src/Interpolations.jl:454 declares type variable N but does not use it.
│ WARNING: method definition for checkbounds at /home/liangxs/.julia/packages/Interpolations/y4lLj/src/Interpolations.jl:457 declares type variable N but does not use it.
│ WARNING: method definition for GriddedInterpolation at /home/liangxs/.julia/packages/Interpolations/y4lLj/src/gridded/gridded.jl:37 declares type variable pad but does not use it.
│ WARNING: method definition for GriddedInterpolation at /home/liangxs/.julia/packages/Interpolations/y4lLj/src/gridded/gridded.jl:60 declares type variable pad but does not use it.
│ WARNING: method definition for interpolate! at /home/liangxs/.julia/packages/Interpolations/y4lLj/src/deprecations.jl:30 declares type variable TWeights but does not use it.
└
┌ VariantCallFormat
│ [ Info: Compiling VCF parser...
└
┌ PyCallUtils
│ WARNING: Method definition getindex(PyCall.PyObject, Union{AbstractString, Symbol}) in module PyCall at /home/liangxs/.julia/packages/PyCall/1gn3u/src/PyCall.jl:349 overwritten in module PyCallUtils at /home/liangxs/.julia/packages/PyCallUtils/GJ7TW/src/PyCallUtils.jl:45.
│ ERROR: Method overwriting is not permitted during Module precompilation. Use __precompile__(false) to opt-out of precompilation.
└
┌ CellScopes
│ WARNING: Method definition getindex(PyCall.PyObject, Union{AbstractString, Symbol}) in module PyCall at /home/liangxs/.julia/packages/PyCall/1gn3u/src/PyCall.jl:349 overwritten in module PyCallUtils at /home/liangxs/.julia/packages/PyCallUtils/GJ7TW/src/PyCallUtils.jl:45.
│ ERROR: Method overwriting is not permitted during Module precompilation. Use __precompile__(false) to opt-out of precompilation.
└

After cs.read_visiumHD(), the same error appeared again:

  1. loading 2um binned data...
    Formatting cell polygons...
    Progress: 100%[==================================================] Time: 0:00:52[39m
  2. 2um binned data loaded!
  3. loading 8um binned data...
    Formatting cell polygons...
    Progress: 100%[==================================================] Time: 0:00:041:39
  4. 8um binned data loaded!
  5. loading 16um binned data...
    Formatting cell polygons...
    Progress: 100%[==================================================] Time: 0:00:02
  6. 16um binned data loaded!
    MethodError: no method matching Union{Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGBA{FixedPointNumbers.N0f8}}}(::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}})

Stacktrace:
[1] convert(::Type{Union{Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGBA{FixedPointNumbers.N0f8}}}}, a::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}})
@ Base ./array.jl:665
[2] convert(::Type{Union{Nothing, Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, Matrix{ColorTypes.RGBA{FixedPointNumbers.N0f8}}}}, x::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}})
@ Base ./some.jl:37
[3] CellScopes.VisiumImgObject(highresImage::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}}, lowresImage::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f16}}, fullresImage::Nothing, detectedTissue::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, alignedImage::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, jsonParameters::Dict{String, Any})
@ CellScopes ~/.julia/packages/CellScopes/XZUoC/src/spatial/sp_objects.jl:249
[4] read_visiumHD(hd_dir::String; min_genes::Vector{Int64}, min_cells::Vector{Int64}, prefix::Nothing, postfix::Nothing, default_bin::String)
@ CellScopes ~/.julia/packages/CellScopes/XZUoC/src/fileio.jl:660
[5] read_visiumHD(hd_dir::String)
@ CellScopes ~/.julia/packages/CellScopes/XZUoC/src/fileio.jl:621
[6] top-level scope
@ In[2]:1

Could you please help me figure out what the problem is? Any clues might be helpful is that the 'import CellScope as cs' produced Precompiling erros and I don't known whether it is associate with the failure of reading samples:
Precompiling CellScopes
✓ StructArrays → StructArraysGPUArraysCoreExt
? PyCallUtils
Info Given CellScopes was explicitly requested, output will be shown live
WARNING: Method definition getindex(PyCall.PyObject, Union{AbstractString, Symbol}) in module PyCall at /home/liangxs/.julia/packages/PyCall/1gn3u/src/PyCall.jl:349 overwritten in module PyCallUtils at /home/liangxs/.julia/packages/PyCallUtils/GJ7TW/src/PyCallUtils.jl:45.
ERROR: Method overwriting is not permitted during Module precompilation. Use __precompile__(false) to opt-out of precompilation.
? CellScopes
1 dependency successfully precompiled in 18 seconds. 575 already precompiled.
2 dependencies failed but may be precompilable after restarting julia
2 dependencies had output during precompilation:
┌ PyCallUtils
│ WARNING: Method definition getindex(PyCall.PyObject, Union{AbstractString, Symbol}) in module PyCall at /home/liangxs/.julia/packages/PyCall/1gn3u/src/PyCall.jl:349 overwritten in module PyCallUtils at /home/liangxs/.julia/packages/PyCallUtils/GJ7TW/src/PyCallUtils.jl:45.
│ ERROR: Method overwriting is not permitted during Module precompilation. Use __precompile__(false) to opt-out of precompilation.
└
┌ CellScopes
│ [Output was shown above]
└
[ Info: Precompiling CellScopes [708d50d9-0a30-4b01-8594-5d83c0e43023]
WARNING: Method definition getindex(PyCall.PyObject, Union{AbstractString, Symbol}) in module PyCall at /home/liangxs/.julia/packages/PyCall/1gn3u/src/PyCall.jl:349 overwritten in module PyCallUtils at /home/liangxs/.julia/packages/PyCallUtils/GJ7TW/src/PyCallUtils.jl:45.
ERROR: Method overwriting is not permitted during Module precompilation. Use __precompile__(false) to opt-out of precompilation.
[ Info: Skipping precompilation since precompile(false). Importing CellScopes [708d50d9-0a30-4b01-8594-5d83c0e43023].
The WebIO Jupyter extension was not detected. See the WebIO Jupyter integration documentation for more information.

Precompiling PyCallUtils
Info Given PyCallUtils was explicitly requested, output will be shown live
WARNING: Method definition getindex(PyCall.PyObject, Union{AbstractString, Symbol}) in module PyCall at /home/liangxs/.julia/packages/PyCall/1gn3u/src/PyCall.jl:349 overwritten in module PyCallUtils at /home/liangxs/.julia/packages/PyCallUtils/GJ7TW/src/PyCallUtils.jl:45.
ERROR: Method overwriting is not permitted during Module precompilation. Use __precompile__(false) to opt-out of precompilation.
? PyCallUtils
[ Info: Precompiling PyCallUtils [02651160-e6c3-11e9-0e2a-551f75d9519e]
WARNING: Method definition getindex(PyCall.PyObject, Union{AbstractString, Symbol}) in module PyCall at /home/liangxs/.julia/packages/PyCall/1gn3u/src/PyCall.jl:349 overwritten in module PyCallUtils at /home/liangxs/.julia/packages/PyCallUtils/GJ7TW/src/PyCallUtils.jl:45.
ERROR: Method overwriting is not permitted during Module precompilation. Use __precompile__(false) to opt-out of precompilation.
[ Info: Skipping precompilation since precompile(false). Importing PyCallUtils [02651160-e6c3-11e9-0e2a-551f75d9519e].
┌ Warning: Using an un-quoted Symbol on the LHS is deprecated. Write :SumValue = ... instead.
└ @ DataFramesMeta ~/.julia/packages/DataFramesMeta/Ga6VV/src/parsing.jl:387
Welcome to use CellScopes.jl!

@HaojiaWu
Copy link
Owner

As mentioned in the link provided (https://github.com/HaojiaWu/CellScopes.jl/tree/dev), if you are using Julia v1.10.4, please use the following command to install:

using Pkg
Pkg.add(PackageSpec(url="https://github.com/bicycle1885/Leiden.jl"))
Pkg.add(PackageSpec(url="https://github.com/HaojiaWu/CellScopes.jl", rev="dev"))

I reinstalled it on a fresh Linux machine, and the methods described above worked for me when reading VisiumHD data.

@liangxs7015
Copy link

As mentioned in the link provided (https://github.com/HaojiaWu/CellScopes.jl/tree/dev), if you are using Julia v1.10.4, please use the following command to install:

using Pkg
Pkg.add(PackageSpec(url="https://github.com/bicycle1885/Leiden.jl"))
Pkg.add(PackageSpec(url="https://github.com/HaojiaWu/CellScopes.jl", rev="dev"))

I reinstalled it on a fresh Linux machine, and the methods described above worked for me when reading VisiumHD data.

I found the problem is that my image is N0f16 and was not supported in this tool. Therefore, I tried inerting 'convert(Matrix[RGB{N0f8}])' functions in the fileio.jl and sp_analysis.jl source codes to convert my image to N0f8, and this problem was successfully solved. However my jupyterlab corrupted after running " cs.normalize_object" for a single HD sample, possibly due to the memory limit?

@HaojiaWu
Copy link
Owner

A typical VisiumHD analysis uses around 30 GB of RAM. If your server has more memory than this, it should be enough. What error you saw when JupyterLab crashed? "segmentation fault"? I am using jupyterlab v4.2.4 with notebook v7.2.1 and it works fine.
You can try copying the code from your notebook into a file (e.g., test_run.jl) and then running it in terminal "julia test_run.jl" to see if the error can be reproduced."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Sophieliula @HaojiaWu @liangxs7015