Jul 9 2008, 10:18 AM
Like to know what is the best way to load huge data every week come from different resources.
1: Huge data
2: Load into one OR multiple tables
3: Data need clean up task also
Like to here from other ETL & Sr.DBA what they are doing in there databases.
Use ETL tools / build sql scripts or any other way...?
Thanks in advance
Jul 9 2008, 10:26 AM
I assume that the data is coming-in as standard flat files?
I do not like the ETL tools, as you can do better if you understand shell scripting in Oracle.
To load data fast, try the Oracle SQL*Loader utility:http://www.dba-oracle.com/t_optimize_sql_l...performance.htm
You can also define the incoming files as external tables, and leave them outside of Oracle:http://www.dba-oracle.com/t_sql_loader_sql...rnal_tables.htm
To learn shell scripting for Oracle, see Jon Emmon's book:http://www.rampant-books.com/book_2007_1_shell_scripting.htm
"Data need clean up task also "
This is called "data scrubbing", and quite easy:http://www.dba-oracle.com/t_online_systems...g_cleansing.htm