Python os.path.exists <=> Blobfuse2 interaction

[FrsECM]

Hi !
I've mounted a storage with blobfuse2 in an azureml compute instance :
It worked well and i can tree my structure :

The problem is that when i perform a "os.path.exists()" in python, it fails because the file is not already cached.

I'm sure that it is existing :

The weard things is that when i restart the job, it seems to be working after more trials....
If i list the directory containing my file and then open the file it work also.
In my configuration it's a blob storage that cause problem.

Do you know if there is a parameter missing in my config in order fuse to "discover" the file structure before using it ?

Here is my yaml file :

# Refer ./setup/baseConfig.yaml for full set of config parameters

allow-other: true

logging:
  type: syslog
  level: log_debug

components:
  - libfuse
  - file_cache
  - attr_cache
  - azstorage

libfuse:
  attribute-expiration-sec: 120
  entry-expiration-sec: 120
  negative-entry-expiration-sec: 240

file_cache:
  path: /tmp/.cache
  timeout-sec: 120
  max-size-mb: 512

attr_cache:
  timeout-sec: 600

azstorage:
  type: block
  account-name: <StorageName>
  endpoint: https://<StorageName>.blob.core.windows.net
  mode: sas
  sas: <SAS Token>


mountall:
  # allowlist takes precedence over denylist in case of conflicts
  container-allowlist:
    - celeba-hq-data

Thanks a lot for your help !

[vibhansa-msft]

When you get the error in your python code, can you try "stat " from your shell just to validate on shell as well you get the error. Also share the debug logs of blobfuse to root-cause this.
When you try file.exists() you are doing it on the mount path and not the local-cache path (/tmp/.cache)?

[FrsECM]

I do the file.exists() on the mount path. Not on the cache.

I tryed "stat" :

In the same terminal if i continue and move to the directory and list it :

It seems that the file is "born" when i list the directory.

[vibhansa-msft]

"--virtual-directory=true" use this in your mount command line option and try again.

[vibhansa-msft]

As this is FNS account, it does not have a concept of a directory. To make directory structure work Blobfuse needs a special directory marker file which gets created only if you use Blobfuse or AzCopy tool to create/copy your data. In case when these marker files are missing we need this command line option to work on the account. Mostly this will fix your problem,

[vibhansa-msft]

If its HNS enable account kindly add "type: adls" to your "azstorage" section in config file.

[FrsECM]

It is exactly what i was searching for !
It works !
Thanks a lot !