Showing Edge Shaped Event Duration in StreamInsight using Debugger

by Allan Mitchell 29 Dec 2010 15:54
Whilst writing some courseware I wanted to be able to see the start and end times of Edge shaped events from within the debugger.  A quick recap on Edge events At the start of the event you do not know the end time and most probably cannot work it out or you should be using one of the other shapes. You enqueue an event (Start Edge) with the start time and payload of the event.  The end time of the event is set to infinity When you see the end edge come through, you enqueue another event (End Edge) with the previous start time and payload and restate the event’s end time.&... [More]

Hopping/Tumbling Windows Could Introduce Latency.

by Allan Mitchell 5 Dec 2010 15:15
This is a pre-article to one I am going to be writing on adjusting an event’s time and duration to satisfy business process requirements but it is one that I think is really useful when understanding the way that Hopping/Tumbling windows work within StreamInsight.  A Tumbling window is just a special shortcut version of  a Hopping window where the width of the window is equal to the size of the hop Here is the simplest and often used definition for a Hopping Window.  You can find them all here public static CepWindowStream<CepWindow<TPayload>> HoppingWindow<TPa... [More]

Linqpad and Autocompletion

by Allan Mitchell 27 Nov 2010 18:22
I have mentioned before about doing development for StreamInsight in Linqpad.  I have it installed on two separate PCs and I have enabled autocompletion on only one of them.  Whilst both versions are an excellent tool, the one with autocompletion enabled is so much easier to use.  After enabling autocompletion you can see I now get parameter listing

Linqpad and StreamInsight

by Allan Mitchell 25 Nov 2010 00:00
Slightly before the announcement of StreamInsight being available for Linqpad I downloaded it from here.  I had seen Roman Schindlauer demonstrate it at Teched and it looked a really good tool to do some StreamInsight dev.   You will need .Net 4.0 and StreamInsight installed. Here’s what you need to do after downloading and installing Linqpad. Add a new connection   The next thing we need to do is install and enable the StreamInsight driver.  Choose to view more drivers.   Choose StreamInsight     Select the driver after install   ... [More]

My Favourite Two Buttons in Denali CTP1 SSIS

by Allan Mitchell 13 Nov 2010 23:03
In SSIS for SQL Server 2005 and SQL Server 2008 when you delete something from the design surface it is gone.  The only real way of getting the deleted item(s) back is to revert to a previous version of the package or to redo the deleted items manually.  Neither of these options is particularly great.  I have made this mistake before and cursed not having CTL+Z and CTL+Y.  Denali changes this.  We can now undo and redo.  Very very welcome.  Well done, finally, the SSIS team.

Using the Script Component as a Conditional Split

by Darren Green 12 Nov 2010 13:46
This is a quick walk through on how you can use the Script Component to perform Conditional Split like behaviour, splitting your data across multiple outputs. We will use C# code to decide what does flows to which output, rather than the expression syntax of the Conditional Split transformation. Start by setting up the source. For my example the source is a list of SQL objects from sys.objects, just a quick way to get some data: SELECT type, name FROM sys.objects type name S syssoftobjrefs F FK_Message_Pa... [More]

Import SSIS Project in Denali CTP1

by Allan Mitchell 11 Nov 2010 22:31
For years Analysis Services has had the ability to take an existing database from a server and reverse engineer it into a BIDS project.  This is extremely useful when all you have is the running instance of the database and the project that created it has long since disappeared.  Reverse engineering has never been a feature of SSIS until now. Let me walk you through the simple steps. The first step is that you obviously have to have a project deployed to an SSIS Catalog.  I will do a video on this soon but in case you can’t wait then my good buddy Jamie Thomson has written it ... [More]

SSIS Field Notes – SQLBits 7 Presentation

by Darren Green 20 Oct 2010 22:35
Here are the slides from my session SSIS Field Notes presented at SQLBits 7 in York earlier this month - SSIS Field Notes – Darren Green.pptx On a similar theme, the video of my session Design patterns for SSIS Performance from is now available. You heard it here first! I know that this because I’ve only just finished updating the SQLBits site with all the videos from SQLBits 6. Hopefully we’ll get them released quicker for SQLBits 7.

Issuing Current Time Increments in StreamInsight (A Practical Example)

by Allan Mitchell 29 Aug 2010 21:02
The issuing of a Current Time Increment, Cti, in StreamInsight is very definitely one of the most important concepts to learn if you want your Streams to be responsive. A full discussion of how to issue Ctis is beyond the scope of this article but a very good explanation in addition to Books Online can be found in these three articles by a member of the StreamInsight team at Microsoft, Ciprian Gerea. Time in StreamInsight Series [More]

SQL Profiler Through StreamInsight Sample Solution

by Allan Mitchell 23 Aug 2010 23:35
In this postI show how you can use StreamInsight to take events coming from SQL Server Profiler in real-time and do some analytics whilst the data is in flight.  Here is the solution for that post.  The download contains Project that reads events from a previously recorded trace file Project that starts a trace and captures events in real-time from a custom trace definition file (Included) It is a very simple solution and could be extended.  Whilst this example traces against SQL Server it would be trivial to change this so it profiles events in Analysis Services. &#... [More]

Additional Columns in StreamInsight Event Flow Debugger

by Allan Mitchell 15 Aug 2010 10:31
This tool is excellent when investigating what is going on in your StreamInsight Streams.  I was looking through the menu items recently and went to Query => Event Fields I found that there were a couple of columns not added by default to the event viewer (Reminds me of the fact that the Variables viewer in SSIS hides columns also) Latency NewEndTime EnqueueTime Here they all are together.     This gives us even more information about what is going on

SQL Bits 7 - 30th September - 2nd October 2010 in York

by Darren Green 12 Aug 2010 15:07
In case you haven't heard we are planning the next SQL Bits event, and today we have released the agenda for Friday & Saturday, a total of 50 sessions covering all aspects of SQL Server with a great selection of speakers. From our recent announcement - ...SQLBits 7 will take place over three days from Thursday September 30th to Saturday October 2nd in York. Day one will be a training day, featuring in-depth full day seminars by leading SQL Server professionals such as Chris Testa-O’Neill and Chris Webb (see [More]

A deadlock was detected while trying to lock variables in SSIS

by Darren Green 10 Aug 2010 22:19
Error: 0xC001405C at SQL Log Status: A deadlock was detected while trying to lock variables "User::RowCount" for read/write access. A lock cannot be acquired after 16 attempts. The locks timed out. Have you ever considered variable locking when building your SSIS packages? I expect many people haven’t just because most of the time you never see an error like the one above. I’ll try and explain a few key concepts about variable locking and hopefully you never will see that error. First of all, what is all this variable locking all about? Put simply SSIS variables have to be l... [More]

Streaming Profiler Data Through StreamInsight

by Allan Mitchell 4 Aug 2010 23:40
Using the Trace Management objects API it is possible to capture the events happening on SQL Server or indeed Analysis Services.  I got to thinking that this would be a good thing to stream out in real time in StreamInsight and do some analysis “In Flight”.  So here is a short video of me doing just that.   Your browser cannot play this video. Learn how to fix this.

AdvanceTimePolicy and Point Event Streams In StreamInsight.

by Allan Mitchell 2 Aug 2010 23:30
There are a number of ways to issues CTIs (Current Time Increments) into your StreamInsight streams but a quite useful way is to do it declaratively on your source factory like this public AdapterAdvanceTimeSettings DeclareAdvanceTimeProperties<TPayload>(InputConfig configInfo, EventShape eventShape) {     return new AdapterAdvanceTimeSettings(         new AdvanceTimeGenerationSettings(configInfo.CtiFrequency, TimeSpan.FromTicks(-1)),         AdvanceTimePolicy.Adjust); } T... [More]

Performing a Left Anti Semi Join in StreamInsight

by Allan Mitchell 18 Jul 2010 09:54
Performing a LEFT OUTER JOIN in TSQL is easy.  The same logic in StreamInsight requires a little more work.  In this presentation I show exactly how you can perform a LEFT OUTER JOIN on your event streams. Your browser cannot play this video. Learn how to fix this.

Talking about StreamInsight in a pub in Maidstone, Kent

by Allan Mitchell 1 Jul 2010 23:41
Come join me for what promises to be a good evening.  18.8.2010.  Sign Up

Talking StreamInsight At Usergroup in London

by Allan Mitchell 1 Jul 2010 23:36
Come join us at the UK SQL Server UG on 14.7.2010 in London

Follow me on Twitter

by Allan Mitchell 28 Jun 2010 18:18
I will be tweeting on all subjects that interest me and come to mind.  Expect to see tweets about StreamInsight, SSIS, Data Mining and Spatial.  Follow me at @allanSQLIS



Creating packages in code - Workflow

by Darren Green 11 Jun 2010 18:08
This is just a quick one prompted by a question on the SSIS Forum, how to programmatically add a precedence constraint (aka workflow) between two tasks. To keep the code simple I’ve actually used two Sequence containers which are often used as anchor points for a constraint. Very often this is when you have task that you wish to conditionally execute based on an expression. If it the first or only task in the package you need somewhere to anchor the constraint too, so you can then set the expression on it and control the flow of execution. Anyway, back to my code sample, here’s a quick scre... [More]


Comment RSS