Skip to content

Spark NLP 4.1.0: Vision Transformer (ViT) is here! The very first Computer Vision pipeline for the state-of-the-art Image Classification task, AWS Graviton/ARM64 support, new EMR & Databricks support, 1000+ state-of-the-art models, and more!
Compare
Choose a tag to compare
@maziyarpanahi maziyarpanahi released this 24 Aug 15:35
· 1053 commits to master since this release
4.1.0
956e2cd

Overview

An Image is Worth 16x16 Words!

For the first time ever we are delighted to announce support for Image Classification in Spark NLP by using state-of-the-art Vision Transformer (ViT) models at scale. This release comes with official support for AWS Graviton and ARM64 processors, new Databricks and EMR support, and 1000+ state-of-the-art models.

Spark NLP 4.1 also celebrates crossing 8000+ free and open-source models & pipelines available on Models Hub. 🎉 As always, we would like to thank our community for their feedback, questions, and feature requests.

⭐ New Features & improvements

  • NEW: Introducing ViTForImageClassification annotator in Spark NLP 🚀. ViTForImageClassification can load Vision Transformer ViT Models with an image classification head on top (a linear layer on top of the final hidden state of the [CLS] token) e.g. for ImageNet. This annotator is compatible with all the models trained/fine-tuned by using ViTForImageClassification for PyTorch or TFViTForImageClassification for TensorFlow models in HuggingFace 🤗 (#11536)

An overview of the ViT model structure as introduced in Google Research’s original 2021 paper

data_df = spark.read.format("image") \
            .load(path="images/")

image_assembler = ImageAssembler() \
            .setInputCol("image") \
            .setOutputCol("image_assembler")

image_classifier = ViTForImageClassification \
    .pretrained() \
    .setInputCols("image_assembler") \
    .setOutputCol("class")

pipeline = Pipeline(stages=[
    image_assembler,
    image_classifier,
])

model = pipeline.fit(data_df)
  • NEW: Support for AWS Graviton/Graviton2 With up to 3x Better Price-Performance. For the first time, Spark NLP supports Graviton and ARM64 (ARMv8 above) processors. (#10939)
  • NEW: Introducing TFNerDLGraphBuilder annotator. TFNerDLGraphBuilder can be used to automatically detect the parameters of a needed NerDL graph and generate the graph within a pipeline when the default NER graphs are not suitable for your training datasets. TFNerDLGraphBuilder supports local, DBFS, and S3 file systems. (#10564)
  • Allow passing confidence scores from all XXXForTokenClassification annotators to NerConverter. It is now possible to access the confidence scores coming from the following annotators in NerConverter metadata (similar to NerDLModel): AlbertForTokenClassification, BertForTokenClassification, DeBertaForTokenClassification, DistilBertForTokenClassification, LongformerForTokenClassification, RoBertaForTokenClassification, XlmRoBertaForTokenClassification, XlnetForTokenClassification, and DeBertaForTokenClassification
  • Introducing PushToHub Python class to easily push public models & pipelines to Models Hub
  • Introducing fullAnnotateImage to existing LightPipeline to support ImageAssembler and ViTForImageClassification annotators in a Spark NLP pipeline. The fullAnnotateImage supports a path to images hosted locally, on DBFS, and S3.
light_pipeline = LightPipeline(model)
annotations_result = light_pipeline.fullAnnotateImage("images/hippopotamus.JPEG")
  • Welcoming a new EMR 6.x series to our Spark NLP family:
    • EMR 6.7.0 (now supports Apache Spark 3.2.1, Apache Hive 3.1.3, HUDI 0.11, PrestoDB 0.272, and Trino 0.378.)
  • Welcoming 3 new Databricks runtimes to our Spark NLP family:
  • Welcoming new AWS Graviton-enabled for Databricks runtime:

Models

Spark NLP 4.1.0 comes with 1000+ state-of-the-art pre-trained transformer models for Image Classifications, Token Classification, and Sequence Classification in many languages.

Featured Models

Model Name Lang
ViTForImageClassification image_classifier_vit_base_patch16_224 en
ViTForImageClassification image_classifier_vit_base_patch16_384 en
ViTForImageClassification image_classifier_vit_base_patch32_384 en
ViTForImageClassification image_classifier_vit_base_xray_pneumonia en
ViTForImageClassification image_classifier_vit_finetuned_chest_xray_pneumonia en
ViTForImageClassification image_classifier_vit_food en
ViTForImageClassification image_classifier_vit_base_food101 en
ViTForImageClassification image_classifier_vit_autotrain_dog_vs_food en
ViTForImageClassification image_classifier_vit_baseball_stadium_foods en
ViTForImageClassification image_classifier_vit_south_indian_foods en
ViTForImageClassification image_classifier_vit_denver_nyc_paris en
ViTForImageClassification image_classifier_vit_CarViT en

Check out 240 (ViT) models on Models Hub - Image Classification

Spark NLP covers the following languages:

English ,Multilingual ,Afrikaans ,Afro-Asiatic languages ,Albanian ,Altaic languages ,American Sign Language ,Amharic ,Arabic ,Argentine Sign Language ,Armenian ,Artificial languages ,Atlantic-Congo languages ,Austro-Asiatic languages ,Austronesian languages ,Azerbaijani ,Baltic languages ,Bantu languages ,Basque ,Basque (family) ,Belarusian ,Bemba (Zambia) ,Bengali, Bangla ,Berber languages ,Bihari ,Bislama ,Bosnian ,Brazilian Sign Language ,Breton ,Bulgarian ,Catalan ,Caucasian languages ,Cebuano ,Celtic languages ,Central Bikol ,Chichewa, Chewa, Nyanja ,Chilean Sign Language ,Chinese ,Chuukese ,Colombian Sign Language ,Congo Swahili ,Croatian ,Cushitic languages ,Czech ,Danish ,Dholuo, Luo (Kenya and Tanzania) ,Dravidian languages ,Dutch ,East Slavic languages ,Eastern Malayo-Polynesian languages ,Efik ,Esperanto ,Estonian ,Ewe ,Fijian ,Finnish ,Finnish Sign Language ,Finno-Ugrian languages ,French ,French-based creoles and pidgins ,Ga ,Galician ,Ganda ,Georgian ,German ,Germanic languages ,Gilbertese ,Greek (modern) ,Greek languages ,Gujarati ,Gun ,Haitian, Haitian Creole ,Hausa ,Hebrew (modern) ,Hiligaynon ,Hindi ,Hiri Motu ,Hungarian ,Icelandic ,Igbo ,Iloko ,Indic languages ,Indo-European languages ,Indo-Iranian languages ,Indonesian ,Irish ,Isoko ,Isthmus Zapotec ,Italian ,Italic languages ,Japanese ,Japanese ,Kabyle ,Kalaallisut, Greenlandic ,Kannada ,Kaonde ,Kinyarwanda ,Kirundi ,Kongo ,Korean ,Kwangali ,Kwanyama, Kuanyama ,Latin ,Latvian ,Lingala ,Lithuanian ,Louisiana Creole ,Lozi ,Luba-Katanga ,Luba-Lulua ,Lunda ,Lushai ,Luvale ,Macedonian ,Malagasy ,Malay ,Malayalam ,Malayo-Polynesian languages ,Maltese ,Manx ,Marathi (Marāṭhī) ,Marshallese ,Mexican Sign Language ,Mon-Khmer languages ,Morisyen ,Mossi ,Multiple languages ,Ndonga ,Nepali ,Niger-Kordofanian languages ,Nigerian Pidgin ,Niuean ,North Germanic languages ,Northern Sotho, Pedi, Sepedi ,Norwegian ,Norwegian Bokmål ,Norwegian Nynorsk ,Nyaneka ,Oromo ,Pangasinan ,Papiamento ,Persian (Farsi) ,Peruvian Sign Language ,Philippine languages ,Pijin ,Pohnpeian ,Polish ,Portuguese ,Portuguese-based creoles and pidgins ,Punjabi (Eastern) ,Romance languages ,Romanian ,Rundi ,Russian ,Ruund ,Salishan languages ,Samoan ,San Salvador Kongo ,Sango ,Semitic languages ,Serbo-Croatian ,Seselwa Creole French ,Shona ,Sindhi ,Sino-Tibetan languages ,Slavic languages ,Slovak ,Slovene ,Somali ,South Caucasian languages ,South Slavic languages ,Southern Sotho ,Spanish ,Spanish Sign Language ,Sranan Tongo ,Swahili ,Swati ,Swedish ,Tagalog ,Tahitian ,Tai ,Tamil ,Telugu ,Tetela ,Tetun Dili ,Thai ,Tigrinya ,Tiv ,Tok Pisin ,Tonga (Tonga Islands) ,Tonga (Zambia) ,Tsonga ,Tswana ,Tumbuka ,Turkic languages ,Turkish ,Tuvalu ,Tzotzil ,Ukrainian ,Umbundu ,Uralic languages ,Urdu ,Venda ,Venezuelan Sign Language ,Vietnamese ,Wallisian ,Walloon ,Waray (Philippines) ,Welsh ,West Germanic languages ,West Slavic languages ,Western Malayo-Polynesian languages ,Wolaitta, Wolaytta ,Wolof ,Xhosa ,Yapese ,Yiddish ,Yoruba ,Yucatec Maya, Yucateco ,Zande (individual language) ,Zulu

The complete list of all 8000+ models & pipelines in 230+ languages is available on Models Hub

New Notebooks

Notebook
Graph Builder
Graph ViTForImageClassification

📖 Documentation

Installation

Python

#PyPI

pip install spark-nlp==4.1.0

Spark Packages

spark-nlp on Apache Spark 3.0.x, 3.1.x, and 3.2.x (Scala 2.12):

spark-shell --packages com.johnsnowlabs.nlp:spark-nlp_2.12:4.1.0

pyspark --packages com.johnsnowlabs.nlp:spark-nlp_2.12:4.1.0

GPU

spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:4.1.0

pyspark --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:4.1.0

M1

spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-m1_2.12:4.1.0

pyspark --packages com.johnsnowlabs.nlp:spark-nlp-m1_2.12:4.1.0

Maven

spark-nlp on Apache Spark 3.0.x, 3.1.x, and 3.2.x:

<dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp_2.12</artifactId>
    <version>4.1.0</version>
</dependency>

spark-nlp-gpu:

<dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp-gpu_2.12</artifactId>
    <version>4.1.0</version>
</dependency>

spark-nlp-m1:

<dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp-m1_2.12</artifactId>
    <version>4.1.0</version>
</dependency>

spark-nlp-aarch64:

<dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp-aarch64_2.12</artifactId>
    <version>4.1.0</version>
</dependency>

FAT JARs

What's Changed

New Contributors

Contributors

@josejuanmartinez @jsl-models @maziyarpanahi @DevinTDHa @agsfer @rpranab @vankov @cayorodriguez @paulk-asert @Ahmetemintek @muhammetsnts @jsl-builder @Cabir40 @diatrambitas @galiph @ahmedlone127 @pabla @Damla-Gurbaz

Full Changelog: 4.0.2...4.1.0

Contributors

paulk-asert, pabla, and 16 other contributors
Assets 2
Loading
0 Join discussion