Vue Js & Papa Parse (Part II): Parsing server side CSVs without webpack / vue-cli bundling / code splitting

In my previous post on Papa Parse and Vue JS, I detailed how to dynamically import CSVs using the vue-cli / webpack.  We covered some issues that we could run into, and finally how to use the file loader correctly with the import statement.

Here is a specific requirement that I ran into:  I needed to dynamically import and parse CSVs that reside inside the project / on the server, but I didn't want webpack to include all of these because I needed to replace the CSVs without having to recompile the whole project

Code Splitting and Bundling

Webpack's code splitting and bundling are some of the best features that webpack provides your project, but it can sometimes get in the way.  There are plenty of articles on the topic, but Webpack will divide up your code into bundles in order to optimize the number of times your project requests data from the server.

The issue: Dynamic imports will include ALL of  your CSVs directly into the bundles.  What happens if you don't want the CSVs in those bundles? 

Explanation: As stated in the previous post, with dynamic imports, webpack doesn't know which file you will load at runtime, so it loads them all. If you have very large CSVs, this can be a big performance hit to your application.  You can check that this happens by check the Network Tab in your browser's developer tools.

Use case: We want to replace CSVs on the server without having to recompile the whole project. How do we do this?

Using fetch() and Vue static assets

After testing several approaches, I abandoned using the file loader since the import() statement will always include these files into the bundles.

The best approach I came up with was loading the CSVs as Vue static assets via fetch() and then using Papa Parse directly on these files. 

Per the Vue docs:

"Any static assets placed in the public folder will simply be copied and not go through webpack. You need to reference them using absolute paths."

The key here is absolute paths. Any relative paths (e.g., "@/folder/subfolder", or "./") will still go through webpack.

My solution

Place all your CSVs in a data folder under the public folder

> npm install papaparse

In top of your .vue <script>:

import * as Papa from "papaparse";
 

In methods:

loadCsv(buttonVal)
   fetch(`/data/${buttonVal}.csv`)
  .then((response) => response.text())
  .then((data) => this.parseAndGo(data));
} 

Notice the absolute path, and that we're not including "public".

Next, you just use PapaParse inside your method (in my case "parseAndGo").  See my previous post for an example, or PapaParse docs

If you check the Network Tab in your browser's developer tools, you'll notice only the file you requested being loaded from the server.

Gotchas

Deployment issue

Since we use absolute paths, when you deploy to your server the paths may not match up.  For instance, if your server is: http://www.mysite.com but your site resides a few directories below: "http://www.mysite.com/sites/myawesomesite/", then your absolute paths will look here: "http://www.mysite.com/data/mycsv.csv" instead of "http://www.mysite.com/sites/myawesomesite/data/mycsv.csv"

Solution: Set publicPath in vue.config.js, and then modify the fetch() statement to use this publicPath

1) In vue.config.js

module.exports = {
  publicPath: process.env.NODE_ENV === 'production'
    ? '/sites/myawesomesite/'
    : '/'
}

2) In the data property of the .vue file, grab the BASE_URL environment variable that you set in vue.config.js

publicPath: process.env.BASE_URL

 

3) Then change the fetch statement to include publicPath:

fetch(`${this.publicPath}data/${buttonVal}.csv)

*note the lack of a forward slash "/" after ${this.publicPath}