Module throws exception on write

[ravi]

The following code throws an exception:

var whdfs = require('webhdfs'),
      hdfs = null;

hdfs = whdfs.createClient(...);
var remotefd = hdfs.createWriteStream('/tmp/writetst.txt');
remotefd.on('error', err => { console.error(err); process.exit(); });
remotefd.on('finish', () => { console.log("Done with remote write."); process.exit(); });
remotefd.write('test');
remotefd.end();

The equivalent using pipe or writeFile works fine:

hdfs.writeFile('/tmp/writetst.txt', err => { console.log(err); process.exit(); });

Here's the exception:

/mnt/alpha/production/srcs/master-latest-git/3p/pkgs-linux/node/lib/node_modules/webhdfs/lib/webhdfs.js:551
      stream.pipe(upload);
            ^
TypeError: Cannot read property 'pipe' of null
    at Request._callback (/mnt/alpha/production/srcs/master-latest-git/3p/pkgs-linux/node/lib/node_modules/webhdfs/lib/webhdfs.js:551:13)

[idibidiart]

Any luck with this? Is streaming write broke? @harrisiirak

[harrisiirak]

@idibidiart thanks for reminding this. Haven't really looked into this. When I have some time I'll do so.

[SavePointSam]

It looks like WebHDFS#write is not yet implemented.

However, if the WebHDFS API is to be a parallel of fs then, I believe, your suggestion wouldn't work anyway. fs.write only supports Buffer or Uint8Array data types. I don't believe string literals are supported, unless String in Node is just an implementation of Uint8Array.

WebHDFS#writeFile supports a Buffer or string as the data param. Relevant code.

It is non-trival to port fs.write to WebHDFS#write as the function signature for write is dramatically different.

For now you should be able to safely use WebHDFS#writeFile as an alternative until this delta is worked on.

[breeze0]

I have the same problem. When I set the parameters in createClient() like this:
var webHDFS = require('webhdfs');
var hdfs = webHDFS.createClient({host: xxx,port:xxx});
var remoteFileStream = hdfs.createWriteStream("/user/upload/");
I will get the exception like this:
stream.pipe(upload);
^
TypeError: Cannot read property 'pipe' of null
If I don't set any parameters in createClient(), this exception will go away but the default host and port is localhost:50070 that is not what I want.
what's wrong?
Thank you.