Unfortunately, the client isn’t the source of record for the data and had very little ability to dictate to the suppliers of the data any of the formatting or layouts of the file. I had a scenario recently where I needed to build a flat file data ETL process for a client. Although this nut has been cracked before, I’d like to add another rather adaptive approach to the problem using T-SQL and SSIS.
It seems you need a process to read the file and parse it before you read the data from the file and attempt to import it. You need to design a process to consume data from comma delimited flat files and as soon as you build the process to read in files of a particular “agreed upon” layout and column order you inevitably encounter a file or two that breaks with the “agreed upon” layout by reordering one or more columns or entirely missing columns of data.
Importing flat files with inconsistent formatting is a classic problem for ETL (Extract Transform and Load) scenarios.
0 kommentar(er)
