Download file from Azure file and save it in temp folder in server by using Node js

 Scenario: 

Download excel file from Azure file system and process it in server through Node Js to generate some meaningful reports.

Pre-requisite:

Here, I am going to assume that as a developer, we are already having below information or tasks ready to write required code to fix the above scenario:

1. A running node js application where we can add required code to read files from Azure file system and download it to our server where node js is running.
2. Install package like: @azure/storage-file-share
3. Required credentials to connect with your Azure storage like AccountName and AccountKey

Note: Here, I am presenting only the way of getting excel files from Azure file system and saving it in your server. To see, how your NodeJs code can read these files from your node js server and process every sheets of every excel file, you can go through my another blog:

Read all sheets of an excel file in Node Js using XLSX package

Implementation:

1) This implementation can be done in following steps:Create azure connection string: Keep your Azure AccountName and AccountKey somewhere safe like in your environment variable or DB. Create connection string by using these two data in following way: 

"azureStorageConnectionString": "DefaultEndpointsProtocol=https;AccountName=YourAccountName;AccountKey=YourAccountKey;EndpointSuffix=core.windows.net",

2) Create a temp directory in your server where your node js code is running so that your node js code can further read and process the downloaded files from this temp directory.

const tempDir = path.join(__dirname, '../../temp_dir_name');

//creating temp folder path 

fs.mkdirSync(tempDir, { recursive: true })

3) Write code for downloading file from Azure file system.

const azurefileClient = serviceClient

    .getShareClient(shareName)

    .rootDirectoryClient.getFileClient(filePathInAzureFileSystem);

    // Get file content from azure file system

    const azureFileResponse = await azurefileClient.download();

    azureFileResponse.readableStreamBody.on('end', () => {

      console.log("File downloaded successfully but yet to be written on nodejs server");

    });

Here, we are creating a read stream i.e. azureFileResponse for reading the file (represented by filePathInAzureFileSystem having the path as well as file name like ext-files/abc/xyz/readMe.xlsx) in binary format from azure file sytem. We are attaching one event handler on this read stream to notify once reading is completed.

4) Now, we are going to create one write stream that will take input from the above read stream and write the data in the temp folder in the same file name in nodejs server.

//creating write stream

    let writer = fs.createWriteStream(tempDir + "/" + "fileName.xlsx");

    writer.on('error', (err) => {

      console.log(err, "Error in writing the downloaded file in nodejs server");

    });

Here, what ever data is being read from azure read stream, that data will be written to the file tempDir/fileName.xlsx file in node js server. Here, we have attached an error event handler with temp file write stream to handle any error if it happens during writing data to temp file.

5) Connect the azure file read stream with temp file write stream through node js pipe functionality so that temp file write stream will receive chucks of data from azure read stream and will keep writing to temp file in nodejs env.

//connecting readStream with write Stream to write downloaded file into temp folder in node js server

    return await pipeline(azureFileResponse.readableStreamBody, writer);

Following is the complete code:

const { ShareServiceClient } = require("@azure/storage-file-share");

const fs = require('fs');

const path = require('path');

const stream = require('stream');

const pipeline = util.promisify(stream.pipeline);

const shareName = config.shareName; 

const tempDir = path.join(__dirname, '../../temp_dir_name');

const azureStorageConnectionStr = "DefaultEndpointsProtocol=https;AccountName=YourAccountName;AccountKey=YourAccountKey;EndpointSuffix=core.windows.net";

const serviceClient =  ShareServiceClient.fromConnectionString(azureStorageConnectionStr);

const downloadFileFromAzureFileSystem = async (filePathInAzureFileSystem) => {

  try {   

    //creating temp folder path 

    fs.mkdirSync(tempDir, { recursive: true })

    const azurefileClient = serviceClient

    .getShareClient(shareName)

    .rootDirectoryClient.getFileClient(filePathInAzureFileSystem);

    // Get file content from azure file system

    const azureFileResponse = await azurefileClient.download();

    azureFileResponse.readableStreamBody.on('end', () => {

      console.log("File downloaded successfully but yet to be written on nodejs server");

    });

    //creating write stream

    let tempFileWriter = fs.createWriteStream(tempDir + "/" + "fileName.xlsx");

    tempFileWriter.on('error', (err) => {

      console.log(err, "Error in writing the downloaded file in nodejs server");

    });

    //connecting readStream with write Stream to write downloaded file into temp folder in node js server

    return await pipeline(azureFileResponse.readableStreamBody, tempFileWriter);

  } catch (error) {

    console.log(error);

    throw error;

  }

}

module.exports = {

  downloadFileFromAzureFileSystem

}