How to Handle Dynamic Rowsets

Sometimes you have a stored procedure that returns a rowset, and you need to insert the rowset into a temporary table in order to perform additional processing. That’s easy, right? You can simply create a temporary table and then use the INSERT-EXECUTE statement in order to insert the rowset into the table.

As an example, let’s create a database, a table with some random data and a stored procedure…

USE
	master;
GO


CREATE DATABASE
	TestDB;
GO


USE
	TestDB;
GO


CREATE TABLE
	dbo.TestTable
(
	Col1	INT				NOT NULL ,
	Col2	NVARCHAR(50)	NULL ,
	Col3	DATETIME2(0)	NOT NULL ,
	Col4	BIT				NOT NULL
);
GO


INSERT INTO
	dbo.TestTable
(
	Col1 ,
	Col2 ,
	Col3 ,
	Col4
)
SELECT TOP (100)
	Col1	= CHECKSUM (NEWID ()) ,
	Col2	= CAST (NEWID () AS NVARCHAR(50)) ,
	Col3	= DATEADD (SECOND , CHECKSUM (NEWID ()) , SYSDATETIME ()) ,
	Col4	= CAST ((ABS (CHECKSUM (NEWID ())) % 2) AS BIT)
FROM
	sys.objects;
GO


SELECT
	Col1 ,
	Col2 ,
	Col3 ,
	Col4
FROM
	dbo.TestTable;
GO


CREATE PROCEDURE
	dbo.TestProcedure
(
	@Col4 AS BIT
)
AS

SELECT
	Col1 ,
	Col2 ,
	Col3 ,
	Col4
FROM
	dbo.TestTable
WHERE
	Col4 = @Col4;
GO


EXECUTE dbo.TestProcedure
	@Col4 = 1;
GO

Now let’s use the INSERT-EXECUTE statement to insert the rowset from the stored procedure to a temporary table…

USE
	TestDB;
GO


CREATE TABLE
	#ResultsTable
(
	Col1	INT				NOT NULL ,
	Col2	NVARCHAR(50)	NULL ,
	Col3	DATETIME2(0)	NOT NULL ,
	Col4	BIT				NOT NULL
);
GO


INSERT INTO
	#ResultsTable
EXECUTE dbo.TestProcedure
	@Col4 = 1;
GO


SELECT
	*
FROM
	#ResultsTable;
GO

But what if you don’t know the structure of the rowset? How can you create the temporary table, if you don’t know the column names and data types, or even the number of columns in the rowset?

I found this question in one of the forums. I wrote my answer there, and then I realized that it’s quite an interesting topic, and people can learn several things form this, so I decided to blog about it here as well…

Before I continue, you might ask why anyone would be in such a situation, where they don’t know the structure of the rowset. They can simply look at the code of the stored procedure, right? Well, sometimes you are required to develop a generic solution that will work with any stored procedure and any rowset. For example, you might want to develop a generic reporting system, where there is a central component, which is responsible for running a report (stored procedure), inserting the results into a temporary table, doing some manipulation on the data, and then sending it to the UI. In this design, people can write more and more reports (stored procedures), and your central component should be able to handle them without knowing the structure of the rowsets in advance.

So now that we understand the need, let’s look at the solution. There are actually 3 solutions to this problem. Well, there are probably more, but I’ll focus on the 3 relatively simple solutions. The first 2 solutions are simple but tricky. The 3^rd solution is more straightforward, but also a bit more complicated.

Ready?

Solution #1 – OPENQUERY

The idea of this solution is to use the SELECT-INTO statement, which lets you create a table and insert data into it based on the result of a SELECT statement. This way, you don’t need to create the table in advance, and you don’t even need to know the structure of the table. The problem is that it only works with a SELECT statement, and not with an EXECUTE statement, so you can’t execute a stored procedure and insert the rowset into a table this way.

This is where the OPENQUERY function becomes handy. This function is useful for executing statements in a remote server (any OLE DB data source) through a linked server. In many cases, it is much more efficient then executing the same statement directly on the linked server using four-part object names.

But another advantage of this function is that it behaves like a table-valued function. This means you can use it in the FROM clause of a query, and this opens many possibilities. One of the possibilities is to use the SELECT-INTO statement.

So the tricky part is to use OPENQUERY with a linked server that references your local instance. This is called a loopback linked server. After you create the linked server, you can write a query to retrieve data from the stored procedure by using the OPENQUERY function with the loopback linked server. Then, all you need to do is to add the INTO clause…

USE
	master;
GO


EXECUTE sys.sp_addlinkedserver
	@server		= N'LoopbackLinkedServer',
    @srvproduct	= N'' ,
    @provider	= N'SQLNCLI' ,
    @datasrc	= N'MYINSTANCE' ,
    @catalog	= N'TestDB';
GO


SELECT
	*
INTO
	#ResultsTable
FROM
	OPENQUERY (LoopbackLinkedServer , N'EXECUTE dbo.TestProcedure @Col4 = 1;');
GO


SELECT
	*
FROM
	#ResultsTable;
GO

Solution #2 – OPENROWSET

This solution is very similar to the previous one, only it uses the OPENROWSET function instead of the OPENQUERY function. The idea is the same: retrieve the rowset from the stored procedure through a SELECT statement, and then add the INTO clause in order to insert the rowset into a temporary table without knowing the structure of the rowset in advance.

The difference between OPENQUERY and OPENROWSET is that the former uses a linked server, which is already defined on the instance, while the latter uses a connection string for an ad-hoc connection. The advantage of the OPENROWSET function is that you don’t need to create a linked server, and you don’t leave any footprints after you’re done. But in order to use the OPENROWSET function, you have to enable the advanced server configuration Ad Hoc Distributed Queries, and this might be a security concern. So based on your needs and policies, you can choose between these two solutions…

USE	
	master;
GO


EXECUTE sys.sp_configure
	@configname		= 'show advanced options' ,
	@configvalue	= 1;
GO


RECONFIGURE;
GO


EXECUTE sys.sp_configure
	@configname		= 'Ad Hoc Distributed Queries' ,
	@configvalue	= 1;
GO


RECONFIGURE;
GO


EXECUTE sys.sp_configure
	@configname		= 'show advanced options' ,
	@configvalue	= 0;
GO


RECONFIGURE;
GO


SELECT
	*
INTO
	#ResultsTable
FROM
	OPENROWSET (N'SQLNCLI' , N'Server= MYINSTANCE;Trusted_Connection=yes;' , N'EXECUTE TestDB.dbo.TestProcedure @Col4 = 1;');
GO


SELECT
	*
FROM
	#ResultsTable;
GO

Solution #3 – sys.dm_exec_describe_first_result_set_for_object

This is a whole different solution based on the sys.dm_exec_describe_first_result_set_for_object function, which is introduced in SQL Server 2012. This function takes an object ID of a stored procedure or trigger as a parameter, and returns the structure of the rowset returned by that object. As its name implies, if there multiple rowsets are returned from the object, then only the structure of the first one is returned.

The nice thing about this function is the way it returns the data types. Instead of the system type ID, the length and all the rest of the metadata, it returns the type name as it should appear in a CREATE TABLE statement, for example. So all you have left to do is to compose that CREATE TABLE dynamically. Then you can insert the rowset into the newly created table using the INSERT-EXECUTE statement…

USE
	TestDB;;
GO


DECLARE
	@Command AS NVARCHAR(MAX);

SET @Command =
	N'
		CREATE TABLE
			#ResultsTable
		(
			';

SELECT
	@Command +=
		name + N' ' + UPPER (system_type_name) + N' ' +
			CASE
				is_nullable
			WHEN 0
				THEN N'NOT NULL'
			WHEN 1
				THEN N'NULL'
			END +
		N' ,
			'
FROM
	sys.dm_exec_describe_first_result_set_for_object (OBJECT_ID (N'dbo.TestProcedure') , 0)
WHERE
	is_hidden = 0
ORDER BY
	column_ordinal ASC;

SET @Command = LEFT (@Command , LEN (@Command) - 7);

SET @Command +=
	N'
		);

		INSERT INTO
			#ResultsTable
		EXECUTE dbo.TestProcedure @Col4 = 1;

		SELECT
			*
		FROM
			#ResultsTable;
	';

EXECUTE sys.sp_executesql
	@statement = @Command;
GO

The problem here is that the temporary table is created inside the dynamic batch, and this is also where it is scoped. It means you can’t reference the table from the external code. One way to overcome this problem is to continue to work on the table inside the scope of the dynamic batch. Another option is to use a regular table rather than a temporary table. But if you need to run multiple instances of your code simultaneously, then you need to somehow generate unique names for the tables you create. And, of course, you need to truncate or drop the table when you’re done. So I guess this solution is not so straightforward after all…

Image may be NSFW.
Clik here to view.

You can download all the code used in this post here:

Image may be NSFW.
Clik here to view.

The post How to Handle Dynamic Rowsets appeared first on .