Comments on: Generating Surrogate Keys

By: Bob

Bob — Sun, 22 Jan 2012 18:37:28 +0000

I have the same problem as AS. In SSIS 2008, I can’t get the key from the script into the OLE DB Destination task. When I put a dataviewer between the Script Transformation task and the OLE DB Destination task, it doesn’t show anything.

Thanks,
Bob

By: bali

bali — Thu, 27 Oct 2011 13:46:35 +0000

Hi,

This seems to be a good solution. But Can we have some custom component to generate incremental value so that the logic can be reused, instead of using SQLTask and script component in each packages.

By: AS

AS — Sat, 04 Jul 2009 14:29:26 +0000

We are running into issues when generating surrogate keys in Sql Server 2008. the code works great with SQl server 2005.

The surrogate key does not get passed to the package.

Any ideas????

By: Joseph

Joseph — Wed, 03 Sep 2008 08:49:33 +0000

The solution is very good. Thanks for it.
But you mentioned that Row Number Transformation task can also be used. I don’t know how. If I want to use incremental data extraction I have to pass the biggest surrogate key (already ditributed to the dimension table) to the Row Number Transformation’s Seed property. But it is not exposed through Expression so I can’t change its value dynamicly. Maybe I miss something or the task is quite inflexible.

By: Renato Buda

Renato Buda — Thu, 26 Jul 2007 08:53:04 +0000

Thanks – this is a great solution and tutorial.

The function should have been in the product to start with.
You have used to opportunity to present a handy introduction to the data transformation script component.

By: khanhmy

khanhmy — Thu, 15 Mar 2007 08:05:00 +0000

Hi Phil,

I’ve already inserted new rows into DimTable using “generating surrogate key”. Everything runs well.

I’ve read the topic: “Checking to see if a record exists and if so update else insert” many times.

Unfortunely, in my case, existing data needs update as historical data and then all changed rows are inserted as current data.

Would you give me some suggestions not to route notEqualChecksum output to staging table?

In my scenario, the notEqualChecksum output first goes straight to the SQLCommand for update after that to staging destination.

For inserting changed rows, I must get the MaxKey and generate surrogate keys again.

I have not much changed data to store ouside the buffer. So, It seem I’m going on the long way.

Hope you can help me! Thank you!

khanhmy

By: Phil Brammer

Phil Brammer — Wed, 14 Mar 2007 14:44:18 +0000

Khanhmy,
The red arrow in this case is coming off of a lookup transformation and indicates NEW records that do not exist in the destination.

Every NEW record will pass through the script transformation and hence obtain a new, unique surrogate key.

I am positive that the select statement will always return the maxValue correctly, because it happens before the dataflow. The select statement is needed to see what the current maximum surrogate key is in the table before we start processing records. No other process touches the destination table (in my case) so yes, it is correct in its operation.

By: khanhmy

khanhmy — Wed, 14 Mar 2007 12:23:35 +0000

Hi Phil,

Generating surrogate key is very very cool!

Thanks alot,

By: khanhmy

khanhmy — Wed, 14 Mar 2007 05:57:06 +0000

Hi Phil,

I wonder if the red arrow contents new dim data to be inserted. And It might comes from the Checksum transformation.

In your scenario, every new record will be assigned a new surrogate key after it has gone thru the Script component, will it?

Are you sure that the select statment always return the maxValue exactly when the data haven’t arrived to the destinestion yet.

thank you for sharing experience,

khanhmy

By: Phil Brammer

Phil Brammer — Wed, 21 Feb 2007 20:29:34 +0000

It’s not really hard as I’ve illustrated.

The issue (and why I hate identity columns) is that the data is more portable when you generate your own surrogate keys. When using identity columns and have referential integrity enforced, moving data from one landscape to another generally breaks that integrity. This is especially important in fact/dimension modeling. When using your own counter as I’ve talked about (I’m not the pioneer in this concept, by the way) it is quite simple and accurate to copy the data from one landscape to another without breaking the integrity.