In SQL Server today, this has to be done with derived tables/CTEs like so:

USE AdventureWorksDW
GO
/* Ignoring the fact that this table
   has a "Status" and an EndDate column */
;WITH CTE AS (
SELECT EmployeeKey,
       FirstName,
       LastName,
       MiddleName,
       Title,
       ROW_NUMBER() OVER (PARTITION BY EmployeeNationalIDAlternateKey
                              ORDER BY StartDate DESC) AS ROW_NUM
  FROM dbo.DimEmployee
)
SELECT EmployeeKey,
       FirstName,
       LastName,
       MiddleName,
       Title
  FROM CTE
 WHERE row_num = 1
 ORDER BY FirstName, LastName
GO

That’s easy enough to understand, of course, but the problem lies in the fact that you have to write a bunch of extra lines of code to accomplish this task.

Teradata has the QUALIFY option which would be represented like this:

 SELECT EmployeeKey,
        FirstName,
        LastName,
        MiddleName,
        Title
   FROM dbo.DimEmployee
QUALIFY ROW_NUMBER() OVER (PARTITION BY EmployeeNationalIDAlternateKey
                               ORDER BY StartDate DESC) = 1

Note how much cleaner that looks. QUALIFY has many advantages, but I’ll leave that as an exercise for you to review. In the mean time, if you would like to see such an option in SQL Server, please vote on Itzik Ben-Gan’s feature request: https://connect.microsoft.com/SQLServer/feedback/details/532474/qualify-request-for-a-new-filtering-clause

Get the Excel document here: SSIS_Prefix_Naming.xlsx (11 KB)

Note that I have not updated this document for 2008 task/component changes, and some of the abbreviations may have been changed from Jamie’s original list of suggestions to suit the needs of my organization at the time.

You can add your own groupings to the Toolbox window by right-clicking anywhere in the toolbox, and selecting “Add Tab”.

For example, in the data flow window, I can add a new tab (or grouping) for Analysis Services Destination components.

1) Right click on the toolbox background and select “Add Tab”

Add Tab

2) Give it a name.

Name Tab

3) Drag components to the new tab.

Dragging Components

4) Repeat as needed.

Repeat

The final results might look something like this:

Final Results

Also, if you find a post here that you’re using to work on an issue in SSIS 2008, let me know if it isn’t working for you. It could be that I don’t have updated content for use in the SSIS 2008 environment. It’s easy enough to update, I just won’t be able to catch all of the items listed here.

One last thing - the annual MVP Summit is coming up in February. If you have something that you’d like for me to bring to the SSIS dev team in terms of feedback, feature requests, bugs, etc…, let me know and I’ll do my best to present them to the team.

What do you think? Is this something you could see value in having implemented?

Connect submission: https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=525805

As always, if you are unsure, it is best to determine the actual version by looking at the files themselves, ie. dtexec.exe, or by confirming the version number in BIDS via Help->About.

While this isn’t a big deal, it is an annoyance. I’ve got the question out to the product team for their response, so stay tuned for more information as it becomes available.

EDIT - Please see Matt Masson’s (SSIS Product Team) comments below.

The answer is none of them. The SSIS Service, quite simply, is responsible for managing the Integration Services interface in SQL Server Management Studio. It enables the ability to import/export packages, view running packages, and view stored packages. It really doesn’t do anything more than that.

Disabling the service will not affect:

• package development (you can develop packages without the service)
• the ability to execute packages (DTEXEC and other executables are responsible for this)
• checkpoint restarts of failed SSIS packages
• the ability for users to query the msdb database for stored packages via SQL
• package execution speed
• SQL Server Agent’s ability to execute packages

The SSIS Service is not cluster-aware, and is in fact not recommended. http://msdn.microsoft.com/en-us/library/ms345193.aspx

EDIT - See Michael Entin’s comment below. The SSIS Service will also cache component/task metadata so that the SSIS runtime engine can poll the cache to see what is installed, which may help speed up package load times however small those gains may be.

These are common questions I see when dealing with “slow” performance of an SSIS package. Sometimes though, the developer of the package does not even understand which component is taking so long to process say, thousands of records in their data flow, so it is understandable why this component may proliferate in many shops. (Sometimes it is needed, of course, but you need to always ask the question - do I need the updates to happen RIGHT NOW?)

If you have a need to update data in another table, while still pushing data from a source to a destination, the OLE DB Command is often used because of the thought that it is part of the data flow, so why not use it.

Read on for more….

The trouble with the OLE DB Command component is that it executes whatever command you have provided for each and every row that passes through it. The data flow is designed to operate on buffers of data, which contain by default around 10,000 rows. If you are simply moving data from one table to another, this concept works really well and quite fast too, I might add.

However, if you need to update data, should you use the OLE DB Command component in that data flow? Are there other options? The answers of course are “maybe” and “yes.”

If you simply need to update data in a table and you don’t care when it happens, then I would recommend not using an OLE DB Command component at all in your data flow. Instead, I recommend you build a staging table on your destination database server, and populate that table with your “update” data. Then, in the control flow, after the data flow successfully executes, you can issue an Execute SQL Task to perform a batch update.

This recommendation is built on the fact that using an OLE DB Command component ensures that your update operations are done row-by-row, rather than in a batch operation.

Why? How much faster is the batch operation compared to the OLE DB Command operation? Well, the answer depends a bit, but in my simple test below, the time to update 5,000 records took 15 minutes using the OLE DB Command component and 4 seconds (yup - SECONDS) to populate a staging table and then perform a batch update.

Here is the setup I used:

CREATE TABLE numbers (
  number int not null
)
GO

insert into numbers (number)
select number from
(
select row_number() over (order by s1.name) as number
  from sysobjects s1
 cross join sysobjects s2
 cross join sysobjects s3
 cross join sysobjects s4
 cross join sysobjects s5
) as numbers
where number between 1 and 1000000;
GO

select number
  from numbers
 where number < 5003;
GO

/*
number
1
2
3
4
5
...
4998
4999
5000
5001
5002
*/

Here is a picture of the first SSIS package.

I used the following SQL for the OLE DB Source component:

SELECT number
  FROM numbers
 WHERE number < 5001
 ORDER BY number DESC

I used the following expression in the Derived Column component to create a new column, new_number:

number + 1

I used the following SQL in the OLE DB Command component:

update numbers
  set number = ?
 where number = ?

In the OLE DB Command “Column Mappings” tab, I mapped the incoming column, new_number, to the Param_0 parameter, and the incoming column, number, to the parameter, Param_1.

I then executed the package. Time to execute: 15 minutes, 43 seconds.

Next I tried the batch update approach. Use the same setup as above, except replace the OLE DB Command component with an OLE DB Destination, and push the two data flow columns to a staging table.

Here is the data flow layout:

Here is the staging table DDL:

CREATE TABLE staging_numbers (
  orig_number int not null,
  new_number int not null
)

Map the data flow column, number, to the orig_number field in the staging table. Map the data flow column, new_number, to the new_number field in the staging table, as shown below:

Then, in the control flow, hook up an Execute SQL Task to the new Data Flow task, so that the Execute SQL Task executes after the Data Flow, as shown below:

The Execute SQL Task SQL is below:

update numbers
  set number = staging_numbers.new_number
from numbers, staging_numbers
where numbers.number = staging_numbers.orig_number

Upon executing the package, the execution time shrank to under five seconds:

So, the moral of this story is that whenever you think you need to use an OLE DB Command component, ask yourself, “Do I *really* need to use this component, or should I use a different approach?”

This book brings together the experience from all corners of SQL Server: Design & Architecture, Development, Administration, Performance Tuning & Optimization, and Business Intelligence. Each section contains unique topics ranging from avoiding common query mistakes, running SQL Server on Hyper-V, to SSIS performance tuning.

The book is available to pre-order here: http://www.sqlservermvpdeepdives.com

(EDIT: Save 50% by purchasing the book from the above site by the end of September when you use the code, pop0928)

All author royalties - and this is what really makes this book stand out - are being donated to War Child International, a charity involved in bringing food, health, and happiness to kids across the world involved in war.

Many thanks to the editors for pushing this book through to completion: Paul Nielsen, Kalen Delaney, Greg Low, Adam Machanic, Paul S. Randal, and Kimberly L. Tripp

For more information on the book, continue reading:

Chapter list:

PART I - DATABASE DESIGN AND ARCHITECTURE
Paul Nielsen, Editor

1. Louis and Paul’s 10 relational database design rules by Paul Nielsen and Louis Davidson
2. SQL Server tools for maintaining data integrity by Louis Davidson
3. Finding functional dependencies by Hugo Kornelis

PART II - DATABASE DEVELOPMENT
Adam Machanic, Editor

4. Set-based iteration: the third alternative by Hugo Kornelis
5. Gaps and islands by Itzik Ben-Gan
6. Error handling in SQL Server and applications by Bill Graziano
7. Pulling apart the FROM clause by Rob Farley
8. What makes a bulk insert a minimally logged operation? by Denis Gobo
9. Avoiding three common query mistakes by Kathi Kellenberger
10. Introduction to XQuery on SQL Server by Michael Coles
11. SQL Server XML frequently asked questions by Michael Coles
12. Using XML to transport relational data by Matija Lah
13. Full text searching by Robert C. Cain
14. Simil: an algorithm to look for similar strings by Tom Van Stiphout
15. LINQ to SQL and the ADO.NET Entity Framework by Bob Beauchemin
16. Table-valued parameters by Don Kiely
17. Build your own index by Erland Sommarskog
18. Getting and staying connected—or not by William Vaughn
19. Extending your productivity in SSMS and Query Analyzer by Pawel Potasinski
20. Why every SQL developer needs to have a tools database by Denis Gobo
21. Deprecation feature by Cristian Lefter
22. Placing SQL Server in your pocket by Christopher Fairbairn
23. Mobile data strategies by John Baird

PART III - DATABASE ADMINISTRATION
Kimberly L. Tripp and Paul S. Randal, Editors

24. What does it mean to be a DBA? by Brad McGehee
25. Working with maintenance plans by Tibor Karaszi
26. PowerShell in SQL Server by Richard Siddaway
27. Automating SQL Server management using Server Management Objects by Allen White
28. Practical auditing in SQL Server 2008 by Jasper Smith
29. My favorite DMVs and why by Aaron Bertrand
30. Reusing space in a table by Joe Webb
31. Some practical issues in table partitioning by Ron Talmage
32. Partitioning for manageability (and maybe performance) by Dan Guzman
33. Faster, smaller, and more energy-efficient backups without indexes by Greg Linwood
34. Using database mirroring to become a superhero! by Glenn Berry
35. The poor man’s SQL Server log shipping by Edwin Sarmiento
36. Some understated changes found in SQL Server 2005 replication by Paul Ibison
37. High performance transactional replication by Hilary Cotter
38. Successfully implementing Kerberos delegation by Scott Stauffer
39. Running SQL Server on Hyper-V by John Paul Cook

PART IV - PERFORMANCE TUNING AND OPTIMIZATION
Kalen Delaney, Editor

40. When is an unused index not an unused index? by Rob Farley
41. Speeding up queries with index covering by Alex Kuznetsov
42. Tracing the deadlock by Gail Shaw
43. How to optimize tempdb performance by Brad McGehee
44. Does the order of columns in an index matter? by Joe Webb
45. Correlating SQL Profiler with Windows Performance Monitor by Kevin Kline
46. Using correlation to improve query performance by Gert-Jan Strik
47. Dr. DMV: how to use Dynamic Management Views by Glenn Berry
48. Query performance and disk I/O counters by Linchi Shea
49. XEVENT: the next event infrastructure for SQL Server by Cristian Lefter

PART V - BI DEVELOPMENT
Greg Low, Editor

50. BI for the relational guy by Erin Welker
51. Unlocking the secrets of SQL Server 2008 Reporting Services by William Vaughn
52. Reporting Services tips and tricks by Bruce Loehle-Conger
53. SQL Server Audit: change tracking or change data capture? by Aaron Bertrand
54. Introduction to SSAS data mining by Dejan Sarka
55. To aggregate or not to aggregate: is there really a question? by Erin Welker
56. Incorporating data profiling in the ETL process by John Welch
57. Expressions in SQL Server Integration Services by Matthew Roche
58. SSIS performance tips by Phil Brammer
59. Incremental loads using T-SQL and SSIS by Andy Leonard