BERT Layer Built-In

[MaximilianSchreff]

This PR introduces the full BERT layer from the BERT transformer architecture to SystemDS as a built-in operation.

This PR is part of a series of PRs to support the BERT architecture in SystemDS. The BERT layer is the component in the BERT architecture. Backward pass will follow, as well as rest of components.

Includes

• Forward pass

Testing:

Added a comprehensive test cases comparing the forward pass results against HuggingFace Transformer Library's implementation for correctness.

• The tests validate:
  • Forward pass against HuggingFace Transformer Library's transformers.models.bert.modeling_bert.BertLayer

[MaximilianSchreff]

@phaniarnab I will add support for the GELU activation here once GELU PR is merged.

[phaniarnab]

@phaniarnab I will add support for the GELU activation here once GELU PR is merged.

The GELU PR is merged now @MaximilianSchreff .

[MaximilianSchreff]

@phaniarnab added support for GELU and also a new test case with GELU activation.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.87%. Comparing base (6c4bffd) to head (1eefe7c).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #2184   +/-   ##
=========================================
  Coverage     71.86%   71.87%           
- Complexity    44707    44715    +8     
=========================================
  Files          1450     1450           
  Lines        169272   169272           
  Branches      32996    32996           
=========================================
+ Hits         121652   121667   +15     
+ Misses        38290    38279   -11     
+ Partials       9330     9326    -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[phaniarnab]

Thanks, @MaximilianSchreff. I will merge this in.

[MaximilianSchreff]

@phaniarnab Added intermediate outputs in outputs since they are required in backward pass. This should be the last patch for this PR and it can be merged :)

[phaniarnab]

@phaniarnab Added intermediate outputs in outputs since they are required in backward pass. This should be the last patch for this PR and it can be merged :)

Thank you @MaximilianSchreff. I will have a look and merge it in.