- Try to perform your data flows in bulk mode instead of row by row.
- Do not sort within Integration Services unless it is absolutely necessary.
- At certain time where using Transact-SQL will be faster than processing the data in SSIS.
- Aggregation calculations such as GROUP BY and SUM.
- These are typically also calculated faster using Transact-SQL instead of in-memory calculations by a pipeline. Delta detection is the technique where you change existing rows in the target table instead of reloading the table.
- Use partitioning on your target table. This way you will be able to run multiple versions of the same package, in parallel, that insert data into different partitions of the same table.
- Make data types as narrow as possible so you will allocate less memory for your transformation.
- Do not perform excessive casting of data types – it will only degrade performance.
- Use the NOLOCK or TABLOCK hints to remove locking overhead.
- If possible, perform your datetime conversions at your source or target databases
- Use a commit size of <5000 to avoid lock escalation when inserting.
- Heap inserts are typically faster than using a clustered index.
- Set Checkpoint So after failure execution start from checkpoint Minimize use of blocking and partially blocking Fast Load Features Optimum Buffering Fast Parse:
- Using fast parse can drastically improve performance of your package when using a flat files source or data conversion transform by basically not validating the columns that you specify.
SSIS, SSAS, SSRS, T-SQL, Optimization, DTS, ETL strategy ,ETL packages, Troubleshooting, Data Modeling, Dimensional Modeling, Data Warehouse Design, Cube Design, OLTP, OLAP, XML
Tuesday, October 25, 2011
Optimizing SSIS Packages
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment