Decision: New design for date-based fetching in the batch pipeline, based on active period (for each patient “active” in care, all the resources will be extracted). We defined an “active” period (the last year), and break down the fetch into 2 parts: (1) any obs from the last year, then (2) for each patient with a history in the active period, all of their history will be downloaded. Why: Avoids search query time-out problems.
Later: Add ability to retrospectively extract resources back in time (re-run pipleline for any time period) in order to update your datawarehouse.
Recording: Analytics Engine Squad - Indiana University