The Trace File Source adapter is a useful addition to your SSIS toolbox. It allows you to read profiler traces stored as .trc files and read them into the Data Flow. From there you can perform filtering and analysis using the power of SSIS. There is no need for a SQL Server connection this just… Continue reading Trace File Source Adapter
Category: Data Flow
Using the Script Component as a Conditional Split
This is a quick walk through on how you can use the Script Component to perform Conditional Split like behaviour, splitting your data across multiple outputs. We will use C# code to decide what does flows to which output, rather than the expression syntax of the Conditional Split transformation. Start by setting up the source.… Continue reading Using the Script Component as a Conditional Split
Expressions
Data Flow Property Expressions Property expressions within the data flow are not immediately obvious. Whilst most pipeline or data flow components do support them for their properties, the expression is not generally exposed through the component user interface, neither is the Expressions property shown properties grid for the component. Property expressions are only available at… Continue reading Expressions
Expression Date Functions
Date Parts Expressions support a range of date related functions such as DATEADD, with the same basic syntax to that found in T-SQL. Whilst the familiarity is very helpful, the difference that catches me out is the format of date part which must be quoted. T-SQL allows this: DATEADD(n, -10, GETDATE()) DATEADD(mi, -10, GETDATE()) DATEADD(minute,… Continue reading Expression Date Functions
Extending SSIS with custom Data Flow components (Presentation)
Download the slides and sample code from my Extending SSIS with custom Data Flow components presentation, first presented at the SQLBits II (The SQL) Community Conference. Abstract Get some real-world insights into developing data flow components for SSIS. This starts with an introduction to the data flow pipeline engine, and explains the real differences between… Continue reading Extending SSIS with custom Data Flow components (Presentation)
Multicast Transform and Threading in SQL Server 2008
The Multicast transform in SSIS 2005 enables us to take 1 input dataset and from it generate n output datasets that are an exact copy of the input dataset which is extremely useful at times and we have used it on a number of projects. The downside is that those outputs are synchronous outputs and… Continue reading Multicast Transform and Threading in SQL Server 2008
Handling different row types in the same file
Sometimes source systems like to send us text files that are a little out of the ordinary. More and more frequently we are seeing people being sent master and detail or parent and child rows in the same file. Handling this in DTS is painful. In SSIS that changes. In this article we'll take you… Continue reading Handling different row types in the same file
Log Events and Pipeline Events
I was about to write about data flow buffer size configuration properties but need to get this one out first so that we can refer to it. The Data Flow task (internally and in the object model also called the 'Pipeline') logs some pretty interesting information that describe the internal scheduler. To see these messages,… Continue reading Log Events and Pipeline Events
Parallel Execution Properties
Adjusting the following properties can have an impact on parallelization during execution: 1. MaxConcurrentExecutables This is a property on the Package. It defines how many tasks can be run concurrently. A value of -1 means the number of processors and when hyperthreading is turned on, it counts the number of logical processors, not physically present… Continue reading Parallel Execution Properties
Generating Surrogate Keys
(By Jamie Thomson) Introduction Surrogate keys are generally considered fundamental building blocks of a data warehouse. They are used as identifiers for dimensional members and enable us to manage slowly changing dimensions. SSIS does not contain a built in component for generating surrogate keys but there is still a mechanism for doing it – the… Continue reading Generating Surrogate Keys