You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
i want to persist the data into the spark application and set the storageLevel to DISK_ONLY. currently i am setting the storage level to spark-defaults.conf like
spark.storage.StorageLevel DISK_ONLY
while running spark application this storage level is set to DISK_ONLY in environment tab on spark ui but when the data persistance is complete the spark ui shows that memory used is something MBs and Disk Used 0.
in spark-shell i am able to set this storage level but i dont know how to set it using Mobius.
so how can i set that storageLevel?
The text was updated successfully, but these errors were encountered:
i want to persist the data into the spark application and set the storageLevel to DISK_ONLY. currently i am setting the storage level to spark-defaults.conf like
spark.storage.StorageLevel DISK_ONLY
while running spark application this storage level is set to DISK_ONLY in environment tab on spark ui but when the data persistance is complete the spark ui shows that memory used is something MBs and Disk Used 0.
in spark-shell i am able to set this storage level but i dont know how to set it using Mobius.
so how can i set that storageLevel?
The text was updated successfully, but these errors were encountered: