Wednesday, October 10, 2012

How to create a change control table?

The purpose of having a control table is to store the last snapshot where ETL stops on each database object. Typical columns in this table could be: object_type, object_name, current_id, current_daytime, refresh_date plus some free text column for comment and notes. It is set up to capture the last image of the unique identifier used to cut off your incremental refresh on each object. For instance, if surrogate key is used as incremental handler, the maximum id number will then be recorded in this table at the end of the process for that individual object. Similarly, if timestamp is what being used to determine fresh data, that piece of information is also kept here in current_daytime column. Once this last event status is captured, the package itself should be smart enough to single out fresh data exactly off your transactional system and process it into data warehouse.

No comments:

Post a Comment