Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding 32bit lossless #158

Merged
merged 45 commits into from
Nov 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
82ee92f
Starting adding support for up to 32bit lossless precision encoding
aous72 Oct 22, 2024
7e5e240
The front end is complete for generic code (not SIMD).
aous72 Oct 29, 2024
5115fb5
Fixes warnings.
aous72 Oct 29, 2024
4d47c40
A bug fix.
aous72 Oct 29, 2024
186e3ed
A bug fix.
aous72 Oct 29, 2024
6993317
Code supports 64 bits, no SIMD, no codeblock encoding/decoding
aous72 Oct 30, 2024
b4e3879
64 bit block encoder/decoder done. No SIMD. Bug in Tiles.
aous72 Nov 1, 2024
163f426
Version bump.
aous72 Nov 1, 2024
c927c04
Changed fail-fast to false
aous72 Nov 1, 2024
3581607
ojph_compress for pfm does not need bit_depth
aous72 Nov 1, 2024
3992c83
update to workflow
aous72 Nov 1, 2024
075ee13
Avoids warnings, and a bug fix -- macos not compiling.
aous72 Nov 1, 2024
abd993d
This should compile on macos
aous72 Nov 1, 2024
630217f
Bug fixes. There are other bugs for 32 lossless.
aous72 Nov 2, 2024
2b6fb15
Reduce warnings.
aous72 Nov 2, 2024
8e1935f
Bug fixes. 32 bit loss should be working now. MacOS may still have …
aous72 Nov 3, 2024
6fb5add
Addresses one warning.
aous72 Nov 3, 2024
8558809
Another attempt to address warning.
aous72 Nov 3, 2024
eafb965
Address another warning
aous72 Nov 3, 2024
5f99c89
Most of the SIMD is done. No wasm yet.
aous72 Nov 5, 2024
1c297b2
A bug fix.
aous72 Nov 5, 2024
c46732f
Removes warnings.
aous72 Nov 5, 2024
da0e69b
Test on Mac-14
aous72 Nov 5, 2024
8114a11
Testing on mac-13
aous72 Nov 5, 2024
c73bfe4
reduces reliance on literals
aous72 Nov 5, 2024
6a4824d
Adds missing limits.h
aous72 Nov 5, 2024
ae28cd0
Let's see without simd.
aous72 Nov 5, 2024
3dc2465
Put back SIMD.
aous72 Nov 5, 2024
4339c8f
A warning fix.
aous72 Nov 5, 2024
9834f17
Incomplete colour_avx2, avx512, and wasm.
aous72 Nov 6, 2024
75259b8
I should have committed these incomplete files.
aous72 Nov 6, 2024
9d2a048
splitting decoder into 32/64 to solve macOS failing tests.
aous72 Nov 6, 2024
e803914
Automated SPqcd to use the LSBs as well.
aous72 Nov 7, 2024
b33083a
AVX2 is done. Bug fix for SPqcd.
aous72 Nov 7, 2024
34e3fc3
Removing two unnecessary lines.
aous72 Nov 7, 2024
f37ae83
All wasm code has been done -- needs extensive tests.
aous72 Nov 8, 2024
1383f9b
All wasm code was tested except for NLT.
aous72 Nov 8, 2024
e86d6c8
This completes the AVX512 code.
aous72 Nov 8, 2024
e96d6da
Added comment to test_executables regarding testing.
aous72 Nov 8, 2024
6386daf
Fixed the SPqcd issue.
aous72 Nov 9, 2024
3292374
This is to address issue #157.
aous72 Nov 9, 2024
c09dfa0
Address warnings in Linux
aous72 Nov 9, 2024
631d781
This modfied raw file processing to support 32bits.
aous72 Nov 9, 2024
576fef9
Changes the messages of ojph_compress.
aous72 Nov 9, 2024
6541f2c
This address codeQL
aous72 Nov 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/workflows/ccp-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ on:
jobs:
build:
strategy:
fail-fast: false
matrix:
include: [
{ system: MacOS, runner: macos-latest },
Expand All @@ -29,6 +30,7 @@ jobs:

build_windows:
strategy:
fail-fast: false
matrix:
include: [
{ system: Windows, runner: windows-latest },
Expand All @@ -46,9 +48,11 @@ jobs:

test:
strategy:
fail-fast: false
matrix:
include: [
{ system: MacOS, runner: macos-latest },
{ system: MacOS-13, runner: macos-13 },
{ system: MacOS-latest, runner: macos-latest },
{ system: Ubuntu-latest, runner: ubuntu-latest },
]
name: ${{ matrix.system }} Test
Expand All @@ -67,6 +71,7 @@ jobs:

test_windows:
strategy:
fail-fast: false
matrix:
include: [
{ system: Windows, runner: windows-latest },
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,11 @@ jobs:

steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
Expand All @@ -64,7 +64,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, Go, Java, or Swift).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2
uses: github/codeql-action/autobuild@v3

# ℹ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
Expand All @@ -77,6 +77,6 @@ jobs:
# ./location_of_script_within_repo/buildscript.sh

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{matrix.language}}"
4 changes: 2 additions & 2 deletions src/apps/common/ojph_img_io.h
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ namespace ojph {
////////////////////////////////////////////////////////////////////////////
// defined elsewhere
class mem_fixed_allocator;
struct line_buf;
class line_buf;

////////////////////////////////////////////////////////////////////////////
//
Expand Down Expand Up @@ -760,7 +760,7 @@ namespace ojph {
const char* fname;
bool is_signed;
ui32 bit_depth, bytes_per_sample;
si32 lower_val, upper_val;
si64 lower_val, upper_val;
ui32 width;
ui8* buffer;
ui32 buffer_size;
Expand Down
34 changes: 16 additions & 18 deletions src/apps/ojph_compress/ojph_compress.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -592,20 +592,25 @@ int main(int argc, char * argv[]) {
".pfm files receive special treatment. Currently, lossy compression\n"
"with these files is not supported, only lossless. When these files are\n"
"used, the NLT segment marker is automatically inserted into the\n"
"codestream. For these files the following arguments can be useful\n"
" -signed a comma - separated list of true or false parameters, one\n"
"codestream when needed, as explained shortly. The following arguments\n"
"can be useful for this file type.\n"
" -signed a comma-separated list of true or false parameters, one\n"
" for each component; for example: true,false,false.\n"
" The sign only affects how values are treated; for negative\n"
" values the standard requires a special non-linear\n"
" transformation. When signed is false, no transformation\n"
" is employed, as we assume all values are 0 or positive.\n"
" When signed is true, the aforementioned transformation is\n"
" employed on negative values only.\n"
" If you are sure that all sample values are positive or 0,\n"
" set the corresponding entry to false; otherwise set it to\n"
" true.\n"
" When a component entry is set to true, an NLT segment\n"
" marker segment is inserted into the codestream.\n"
" The NLT segment specifies a non-linear transform that\n"
" changes only negative values, producing better coding\n"
" efficiency.\n"
" The NLT segment marker might be less supported in other\n"
" encoders.\n"
" -bit_depth a comma-separated list of bit depth values, one per \n"
" component; for example: 12,10,10.\n"
" Floating value numbers are treated as integers, and they\n"
" are shifted to the right, keeping only the specified\n"
" number of bits. Note that a bit depth of 28 upwards is not\n"
" number of bits. Up to 32 bits (which is the default) are\n"
" supported.\n"

"\n";
Expand Down Expand Up @@ -768,10 +773,6 @@ int main(int argc, char * argv[]) {
assert(num_comps == 1 || num_comps == 3);
siz.set_num_components(num_comps);

if (bit_depth[0] == 0)
OJPH_ERROR(0x01000091,
"-bit_depth must be specified (this is temporary only).\n");

if (bit_depth[0] != 0) // one was set
if (num_bit_depths < num_comps) // but if not enough, repeat
for (ojph::ui32 c = num_bit_depths; c < num_comps; ++c)
Expand Down Expand Up @@ -840,11 +841,8 @@ int main(int argc, char * argv[]) {
nlt.set_type3_transformation(c, true);
}
else
OJPH_ERROR(0x01000093, "The support for pfm image is not "
"complete; I need to figure how to modify the interface "
"to better support the exchange of floating point data. "
"Feeding float point data is not supported yet, unless it "
"is for lossless compression.");
OJPH_ERROR(0x01000093, "We currently support lossless only for "
"pfm images; this may change in the future.");

codestream.set_planar(false);
if (profile_string[0] != '\0')
Expand Down
Loading