Downloading a file over HTTP the SSIS way

by Darren Green 2 Jul 2009 18:14
This post shows you how to download files from a web site whilst really making the most of the SSIS objects that are available. There is no task to do this, so we have to use the Script Task and some simple VB.NET or C# (if you have SQL Server 2008) code. Very often I see suggestions about how to use the .NET class System.Net.WebClient and of course this works, you can code pretty much anything you like in .NET. Here I’d just like to raise the profile of an alternative. This approach uses the HTTP Connection Manager, one of the stock connection managers, so you can use configurations and prop... [More]

Logging connection strings

by Darren Green 10 Jun 2009 21:44
If you some of the dynamic features of SSIS such as package configurations or property expressions then sometimes trying to work out were your connections are pointing can be a bit confusing. You will work out in the end but it can be useful to explicitly log this information so that when things go wrong you can just review the logs. You may wish to develop this idea further and encapsulate such logging into a custom task, but for now lets keep it simple and use the Script Task. The Script Task code below will raise an Information event showing the name and connection string for a connection... [More]

Integrating Data Mining into your BI Solution (Presentation)

by Allan Mitchell 9 Jun 2009 08:23
I recently gave a live meeting presentation to the UK User Group on Integrating Data Mining into your BI Solution.  In it I talk about and demo ways of using your data mining models inside Integration Services, Analysis Services and Reporting Services.  This is the first in a series of presentations I will be doing for the UG as I try to get the word out that Data Mining can be for the masses. You can download my deck and my line meeting recording from here.

SSIS Tips & Tricks (Presentation)

by Darren Green 21 May 2009 09:01
This has been a rather well used presentation title but it does allow a certain degree of flexibility, and we covered a good range of topics in my session at the UK SQL Server User Group in Cambridge last night. Thanks to all who attended. Here is the rather limited slide deck and the all important demo packages for download as promised. For reference, high level topics covered were BIDS Helper Inserts and Updates Transactions Script Debugging Data Flow Checkpoints I’ll update the post with a link to the Live Meeting recording when I get it. Presentation & Demo ... [More]

Creating packages in code – Execute SQL Task

by Darren Green 19 May 2009 08:30
The Execute SQL Task is for obvious reasons very well used, so I thought if you are building packages in code the chances are you will be using it. Using the task basic features of the task are quite straightforward, add the task and set some properties, just like any other. When you start interacting with variables though it can be a little harder to grasp so these samples should see you through. Some of these more advanced features are explained in much more detail in our ever popular post The Execute SQL Task, here I’ll just be showing you how to implement them in code. The abbreviated cod... [More]

July SQL Server UG Event in Manchester

by Allan Mitchell 18 May 2009 16:18
I will be speaking at the SQL Server UK User Group event in Manchester on 16.07.2009.  I am going to be talking about data mining again and how it isn’t all statistics and people with PhDs from Oxford.  Come join me and the excellent Chris Testa-O’Neill.  More details and registration can be found here

UK SQL Server User Group Events (June)

by Allan Mitchell 13 May 2009 07:12
There are two events of note for the SQL Server User Group in June.  The first is a Live Meeting event with myself on 04.06.2009.  I am going to be looking at how to integrate Data Mining into your BI solution.  I will be looking at putting DM into SSIS, SSAS and SSRS.  It will be very demo oriented.  You can register for the event here The second event is an event at Microsoft Reading on 10.06.2009.  The evening will be a BI/Data Mining event.  Chris Webb and myself are organizing it and  we want speakers.  We would love to see new faces up the... [More]

UK SQL Server User Group Event (May)

by Allan Mitchell 13 May 2009 07:02
Our very own Darren Green is speaking at a UK user group event in Cambridge (UK) on 20.05.2009.  He will be speaking on Integration Services.  Peter Blackburn will also be there and what he doesn’t know about SSRS isn’t worth knowing.  It promises to be a good night.  We would love to see as many people there as possible so head over to the UK User Group site and register. Register Here

Searching for tasks with code – Executables and Event Handlers

by Darren Green 12 May 2009 09:40
Searching packages or just enumerating through all tasks is not quite as straightforward as it may first appear, mainly because of the way you can nest tasks within other containers. You can see this illustrated in the sample package below where I have used several sequence containers and loops. To complicate this further all containers types, including packages and tasks, can have event handlers which can then support the full range of nested containers again. Towards the lower right, the task called SQL In FEL also has an event handler not shown, within which is another Execute SQL Task,... [More]

Maintaining packages with code - Adding a property expression programmatically

by Darren Green 7 May 2009 14:15
Every now and then I've come across scenarios where I need to update a lot of packages all in the same way. The usual scenario revolves around a group of packages all having been built off the same package template, and something needs to updated to keep up with new requirements, a new logging standard for example.You'd probably start by updating your template package, but then you need to address all your existing packages. Often this can run into the hundreds of packages and clearly that's not a job anyone wants to do by hand. I normally solve the problem by writing a simple console applicat... [More]

Creating packages in code - Package Configurations

by Darren Green 2 Mar 2009 10:00
Continuing my theme of building various types of packages in code, this example shows how to building a package with package configurations. Incidentally it shows you how to add a variable, and a connection too. It covers the five most common configurations: Configuration File Indirect Configuration File SQL Server Indirect SQL Server Environment Variable  For a general overview try the SQL Server Books Online Package Configurations topic. The sample uses a a simple helper function ApplyConfig to create or update a configuration, although in the example we will o... [More]

Creating packages in code - Flat File Source to OLE-DB Destination (SQL Server)

by Darren Green 23 Feb 2009 13:00
This code sample programmatically creates a package that imports a text file into SQL Server, with a Flat File Source and the OLE-DB Destination. This shows how you can leverage the SSIS engine to write your own data import tool for example, but beware importing files is not as simple as it may seem. When you build a similar package in the designer and and select your file, you make some choices about the file format. It seems quite simple, but there is actually some quite complex thinking behind the scenes to parse the file and make suggestions on the file format for columns and their data ... [More]

SSIS Performance Tuning Seminar (1 day)

by Allan Mitchell 18 Feb 2009 12:29
I am going to be delivering a one day tuning seminar for SSIS (SQL Server 2005).  I am doing it in conjunction with the UK User Group.  I will be covering as much as I am able on how to look at your package with performance in mind.  I will show you how to get more from the data flow engine and also how to avoid common performance slow downs.  The location we have chosen is beautiful and it should be really good day. if you are interested then shoot over to here Perf Course

Attunity Oracle CDC Solution for SSIS - Beta

by Allan Mitchell 18 Feb 2009 12:23
We in no way work for Attunity but we were asked to test drive a beta version of their Oracle CDC solution for SSIS.  Everybody should know that moving more data than you need to takes too much time and uses resources that may better be employed doing something else.  Change data Capture is a technology that is designed to help you identify only the data that has had something done to it and you can therefore move only what is needed.  Microsoft have implemented this exact functionality into SQL server 2008 and I really like it there.  Attunity though are doing it on Oracle... [More]

SQLBits IV Conference

by Allan Mitchell 9 Feb 2009 16:10
I am very proud to announce that I will be presenting at this free conference in Manchester on 28.03.2009.  My session will be on mixing SSIS and Data Mining.  It is a free conference organized by people giving up their own time and is aimed at the community.  Everybody I know who has ever been always says that they enjoyed it.   For more details go to www.SQLBits.com

Techfest 2009 Conference Poland

by Allan Mitchell 8 Feb 2009 16:18
I am going to be speaking at this excellent 3 day conference being held in Southern Poland on 3-5 April 2009.  I will be speaking on SSIS. http://www.techfest2009.com/default.aspx

C2C Conference In Poland

by Allan Mitchell 8 Feb 2009 16:10
I am proud to say that I have been chosen to speak at the Communities to Communities Conference in Poland.  The conference is free to attend and is being held in Warsaw, Poland.  I will be delivering a session on SSIS and performance.  Hope to see you all there.

PASS European Conference

by Allan Mitchell 8 Feb 2009 15:59
Well it’s that time of year again and I am happy to say I have been chosen to deliver a couple of sessions at this year’s conference.  The PASS European Conference, on April 22nd-24th in Neuss in Germany is a great conference and I had fun last year. I will be doing a pre-conference seminar with my good friend Chris Webb.  It is a beefed up version of a presentation he gave at PASS in the United States last year.   Me, being the SSIS expert, I will be covering that side of things and rewriting Chris’s packages so they're rather more robust; allowing Chris to concentrate on... [More]

CreationName for SSIS 2008 and adding components programmatically

by Darren Green 2 Feb 2009 16:14
If you are building SSIS 2008 packages programmatically and adding data flow components, you will probably need to know the creation name of the component to add. I can never find a handy reference when I need one, hence this rather mundane post. See also CreationName for SSS 2005. We start with a very simple snippet for adding a component: // Add the Data Flow Task package.Executables.Add("STOCK:PipelineTask"); // Get the task host wrapper, and the Data Flow task TaskHost taskHost = package.Executables[0] as TaskHost; MainPipe dataFlowTask = (MainPipe)taskHost.InnerObject... [More]

Creating packages in code - OLE-DB Source to Flat File File Destination

by Darren Green 29 Jan 2009 18:02
This code sample programmatically creates a package with an OLE-DB Source and a Flat File Destination, and the resulting package exports data from SQL Server to a CSV file. The finished package just has the one Data Flow Task shown below.   The code creates the package, configures the task, and components, then saves the package to disk, useful for checking the package and testing, before finally executing. namespace Konesans.Dts.Samples { using System; using Microsoft.SqlServer.Dts.Runtime; using Microsoft.SqlServer.Dts.Pipeline.Wrapper; using RuntimeWrapper ... [More]