timeSeriesListMerge produce undefined value

[renhax]

Hi,
I'm having issue when to merge two time series with different time range, the column has less time range will have some undefined value. It will cause some issue when I draw the graph, is there any way to fill those undefined value to null or 'N/A', or remove after merge completely. I notice fill doesn't support that.

Thanks,
Renhao

[gingerizer]

I've also observed this and attempted to use timeSeries.fill(options) in an attempt to replace the undefined values with zero. The fill doesn't seem to work however, likely because undefined is not currently considered missing or invalid. Perhaps I'm using fill incorrectly? In either case, the documentation on the fill function isn't clear on the definition of missing or invalid, or on how to replace undefined values with zeros.

I'm currently forced to manually correct the point array, and create a new TimeSeries which is less than ideal.

[pjm17971]

Can either of you post some code that shows the problem and I can look into what the best solution is, or add a enhancement request to cove that use case? Also, what version of Pond are you using?

Currently fill isn't going to help you, it's for actual missing values along a given period. i.e. the value actually has to be missing. It might be a fairly easy change to optionally allow fill to support null/undefined/NaN type invalid values as missing.

[gingerizer]

Created this example quickly using data from react-timeseries-charts examples. As I was creating this I realized I forgot to mention that this happens in the case when the series being merged together don't have common column names so they can't be concat'd together by the merge. Obviously in the example I've created it doesn't make sense to change the column names but in my real world case it makes sense. I have distinct series that I want to display on a stacked bar chart which requires a single series with independent columns.

In the example, the second time series is missing the first 5 points from the original series. The first 5 points in the array contain undefined values for the cropped series column.

I.e. merged time series

const data = [
  ["2017-01-24T00:00", 0.01],
  ["2017-01-24T01:00", 0.13],
  ["2017-01-24T02:00", 0.07],
  ["2017-01-24T03:00", 0.04],
  ["2017-01-24T04:00", 0.33],
  ["2017-01-24T05:00", 0.2],
  ["2017-01-24T06:00", 0.08],
  ["2017-01-24T07:00", 0.54],
  ["2017-01-24T08:00", 0.95],
  ["2017-01-24T09:00", 1.12],
  ["2017-01-24T10:00", 0.66],
  ["2017-01-24T11:00", 0.06],
  ["2017-01-24T12:00", 0.3],
  ["2017-01-24T13:00", 0.05],
  ["2017-01-24T14:00", 0.5],
  ["2017-01-24T15:00", 0.24],
  ["2017-01-24T16:00", 0.02],
  ["2017-01-24T17:00", 0.98],
  ["2017-01-24T18:00", 0.46],
  ["2017-01-24T19:00", 0.8],
  ["2017-01-24T20:00", 0.39],
  ["2017-01-24T21:00", 0.4],
  ["2017-01-24T22:00", 0.39],
  ["2017-01-24T23:00", 0.28]
];

// Series with data for full time range
const series = new TimeSeries({
  name: "hilo_rainfall",
  columns: ["index", "precip"],
  points: data.map(([d, value]) => [
    Index.getIndexString("1h", new Date(d)),
    value
  ])
});

// Series with a different Time Range from Series 1
const seriesCropped = new TimeSeries({
  name: "hilo_rainfall_cropped",
  columns: ["index", "precip_cropped"],
  points: data
    .slice(5)
    .map(([d, value]) => [Index.getIndexString("1h", new Date(d)), value])
});

const mergedSeries = TimeSeries.timeSeriesListMerge({
  name: "MergedSeries",
  seriesList: [series, seriesCropped]
});

// Result of the merged Series ....
mergedSeries = { 
   name: "MergedSeries", 
   utc:  true
   columns:  [index, percip, percip_cropped]
   points:  [ ["1h-412565", 0.01, undefined] , 
            ...]
  }

[gingerizer]

Yes, it would be nice if fill would treat undefined as a missing value. Just for reference, to fix the merged series points I did the following using lodash:

const fixedSeriesPoints = mergedSeries.toJSON().points.map(point => {
   return point.map(p => {
      return _.isUndefined(p) ? 0 : p;
   });
});

const fixedSeries = new TimeSeries({
   name: "MergedSeries",
   columns: mergedSeries.toJSON().columns,
   points: fixedSeriesPoints 
});