Examples

This is a basic example using the HDFS2FileSink operator to write output to a Hadoop filesystem deployed on IBM Cloud (IBM Analytics Engine). 


 stream<rstring fileName, uint64 size> HdfsSink= HDFS2FileSink(Input){ 
      param 
         file          : "/user/clsadmin/myfile.txt"; 
         hdfsUri       : "webhdfs://server_host_name:port"; 
         hdfsUser      : "clsadmin"; 
         hdfsPassword  : "IAEPassword"; 
 }

This example uses the HDFS2FileSink operator to write tuples to output files that have names like output0.txt . SSL certificate validation is enabled. 


 stream<PersonSchema> In = FileSource(){ 
     param 
         file          : "Input.txt"; 
 } 


 stream<rstring PersonSchemString> SingleStringIn = Functor(In){ 
    output 
        SingleStringIn : PersonSchemString =(rstring) In ; 
 } 


 () as txtSink = HDFS2FileSink(SingleStringIn){ 
      param 
         file             : "output%FILENUM.txt"; 
         bytesPerFile     : (int64)(16*1024); 
         hdfsUri          : "webhdfs://server_host_name:port"; 
         hdfsUser         : "clsadmin"; 
         hdfsPassword     : "IAEPassword"; 
         keyStorePassword : "storepass"; 
         keyStorePath     : "etc/store.jks"; 
 }