Bluedog67

Random Thoughts About SQL Server and My Dog Blue

About Bluedog67

My name is Stephen Horne. I am a well-trained owner of an Australian Cattle Dog named Blue. I also develop software mainly using SQL Server, ASP.NET, and C#. I love data, databases, T-SQL, and turning raw data into actionable information. My plan for this blog is to write about SQL Server and sometimes my dog Blue. Please contact me at stephen at bluedog67 dot com. Follow me on Twitter at bluedog67.

SQL Server Logon Trigger: Restricting Access By IP To Your SQL Server Instance

On Twitter today @anonythemouse tweeted the following "any good #sql links for login triggers to block hostnames from logging in? #sqlhelp". Looked like an interesting question - I thought I would give the question a try and see if I could come up with an answer.

Following scripts are based upon researching the following links from the SQL Server 2008 Books Online (BOL):

Below are two possible options to restrict access by IP to SQL Server (developed and tested on SQL Server 2008 R2 Ent Eval Ed CTP Nov only):

Blacklist Option (Clients that are blocked):

CREATE TRIGGER tr_logon_hostname_blacklist
ON ALL SERVER WITH EXECUTE AS 'sa'
FOR LOGON
AS
BEGIN
  DECLARE @ClientHost nvarchar(max);
  SELECT @ClientHost = EVENTDATA().value('(/EVENT_INSTANCE/ClientHost)[1]','nvarchar(max)');
  -- ClientHost gives IP except if the connecting to SQL Server from instance machine.
  -- Do NOT put '<local machine>' in this otherwise you will block client access from instance machine.
  IF @ClientHost IN ('10.0.2.53'
                    ,'10.0.2.54'
                    ,'10.0.2.55'
                    ,'10.0.2.56')
    ROLLBACK;
END;

Whitelist Option (Clients that are allowed access):

CREATE TRIGGER tr_logon_hostname_whitelist
ON ALL SERVER WITH EXECUTE AS 'sa'
FOR LOGON
AS
BEGIN
  DECLARE @ClientHost nvarchar(max);
  SELECT @ClientHost = EVENTDATA().value('(/EVENT_INSTANCE/ClientHost)[1]','nvarchar(max)');
  -- ClientHost gives IP except if the connecting to SQL Server from instance machine.
  IF @ClientHost NOT IN ('<local machine>' -- client is on the instance box (Do NOT change or remove!)
                        ,'10.0.2.54'
                        ,'10.0.2.55'
                        ,'10.0.2.56')
    ROLLBACK;
END;

Please note that the '<local machine>' string should NOT be modified or removed from the Whitelist trigger - you want to continue to be able to connect to your SQL Server instance from the instance box itself correct!?! Before running these queries please make sure that you have an open connection in another Query window in SSMS with the following script to drop the Logon Trigger just in case you make a mistake. You do not want to break your instance!

DROP TRIGGER tr_logon_hostname_blacklist
ON ALL SERVER;

DROP TRIGGER tr_logon_hostname_whitelist
ON ALL SERVER;

Fair Warning: Please note that Logon Triggers can be potentially instance breaking things. Especially if they include a ROLLBACK as the two scripts above include. Strongly recommend testing retesting and reretesting scripts fully on a Test box before even considering getting near a Production box. Make sure you have a separate open query window in SSMS or preferrably an already open SQLCMD session with the DROP TRIGGER script at the ready just in case you break things.

Please share any feedback or ideas that you may have. Thanks!

Categories: SQL Server
Permalink | Comments (1) | Post RSSRSS comment feed

Yet Another FizzBuzz Solution: This Time Using T-SQL CTE (Common Table Expression) and spt_values

Insomnia triumphs over all. Sleep schedule is whacked. Saw this tweet from Jeff Atwood (Blog|Twitter) that referenced his post entitled "Why Can't Programmers.. Program?" located here. I thought I would accept the FizzBuzz challenge and try and program something in Transact-SQL.

Following is my attempt developed and tested on SQL Server 2008 Dev Ed. This solution uses a recursive CTE (Common Table Expression). You can find tons more about recursive CTEs here.

Please note that I know that CASE statement logic could be simplified - you can rely on WHEN expression order for evaluating which THEN clause is used for a given row. My preference is that with IF ELSE ELSE structures that the options be mutually exclusive.

;WITH cteCounter(number)
AS
(
SELECT 1 number
UNION ALL
SELECT (cteCounter.number + 1)  number
FROM cteCounter
WHERE (cteCounter.number + 1) <= 100
)
-- Switch out the SELECT clauses to see basic recursive CTE behavior and possible FizzBuzz solution.
-- SELECT cteCounter.number
SELECT CASE WHEN ((cteCounter.number % 3 = 0) AND (cteCounter.number % 5 = 0)) THEN 'FizzBuzz'
            WHEN ((cteCounter.number % 3 = 0) AND (cteCounter.number % 5 <> 0)) THEN 'Fizz'
            WHEN ((cteCounter.number % 3 <> 0) AND (cteCounter.number % 5 = 0)) THEN 'Buzz'
            ELSE CAST(cteCounter.number AS varchar(8)) END number
FROM cteCounter
ORDER BY cteCounter.number;

Dave Ballantyne (Blog|Twitter) publicly shamed me on Twitter for using a CTE to generate a range of number - thanks Dave <grin>! He rightly suggested using master.dbo.spt_values instead. I swore I would never use a CTE to implement a range of numbers! Following is the updated code.

SELECT CASE WHEN ((V.number % 3 = 0) AND (V.number % 5 = 0)) THEN 'FizzBuzz'
            WHEN ((V.number % 3 = 0) AND (V.number % 5 <> 0)) THEN 'Fizz'
            WHEN ((V.number % 3 <> 0) AND (V.number % 5 = 0)) THEN 'Buzz'
            ELSE CAST(V.number AS varchar(8)) END number
FROM master.dbo.spt_values V
WHERE V.type = 'P'
AND V.number between 1 and 100
ORDER BY V.number;

Categories: SQL Server
Permalink | Comments (1) | Post RSSRSS comment feed

Fiscal Week Scalar Value Function

This post is a quick and dirty solution to a question that came up on Twitter on the #sqlhelp hash tag from Barnaby Self (Blog|Twitter). @B4PJS wrote up a related blog post here. Please note that this possible solution has some potentially significant limitations: First, the possible solution assumes that the Fiscal Year begins on same date every year (some companies like Cisco have 52 week Fiscal Years). Second, the solution assumes that the Fiscal Year start date is Day 1 of Week 1 regardless of the day of the week. In other words, this solution is NOT affected by the SET DATEFIRST command. If SET DATEFIRST needs to impact Fiscal Week determination, this solution will need to be modified. Third, using scalar-valued UDFs (User Defined Functions) can cause significant performance problems if used in WHERE or GROUP BY clauses (nod to @DaveBally) - alternate solutions may yield better results. A possible better performing solution might be using the UDF described below in conjunction with either a persisted computed column or an indexed view.

-- Developed and tested only on SQL Server 2008 Dev Ed.
-- Assumes Fiscal Year begins July 1st.
-- Assumes day 1 of week 1 is July 1st regardless of day of week.
CREATE FUNCTION dbo.udf_GetFiscalYearWeek(@InputDate date)
RETURNS int
AS
BEGIN
  DECLARE @Result int
  
  DECLARE @FiscalYear int;
  -- Following logic will need to change if Fiscal Year start date different from YYYY0701.
  IF (MONTH(@InputDate) >= 7)
  BEGIN
    SET @FiscalYear = YEAR(@InputDate) + 1;
  END
  ELSE
  BEGIN
    SET @FiscalYear = YEAR(@InputDate);
  END

  DECLARE @FiscalYearStartDate date;
  -- Change '0701' as required.
  SET @FiscalYearStartDate = CAST((CAST((@FiscalYear - 1) AS varchar(4)) + '0701') AS DATE);

  DECLARE @FiscalYearWeek int;
  SET @FiscalYearWeek = DATEDIFF(d, @FiscalYearStartDate, @InputDate) / 7 + 1;

  SET @Result = @FiscalYearWeek;
  
  RETURN @Result
END
GO

Thanks for reading. Any feedback you have will be appreciated.

Categories: SQL Server
Permalink | Comments (7) | Post RSSRSS comment feed

T-SQL Tuesday #002: Challenge: Create Unbeatable Tic-Tac-Toe Game in T-SQL

This is my post for Adam Machanic's T-SQL Tuesday #002. Thanks for reading! Feedback is appreciated.

Can you create a game in SQL Server?

A few years back at the Austin Game Developers Conference in Texas I got to meet and talk with the CEO and the lead database developer of the Icelandic company (CCP Games) that created EVE Online - a scifi MMORPG. Some nice and very sharp guys. They had been eCommerce web/database developers plus gamers. They looked at MMORPGs and said to themselves, "these are just huge/high transaction database applications - we can do this!" And that is what they did. They designed and built the database in SQL Server and wrote tons of stored procedures. They made it so you could "play" the game by calling the stored procedures. And then they got a few artists and a couple of programmers to build a beautiful front end for their database application <grin>. The rest is history - EVE Online exploded in popularity and the company became very successful. They proved you can write a game in SQL Server.

Now back to the original question: Can you create a game in SQL Server?

That is The Challenge: Write a game in SQL Server. Not just any game but an Unbeatable Tic-Tac-Toe Game in SQL Server using Transact-SQL.

I have done this in C, C++ and Visual Basic, but never in Transact-SQL. It should be possible! I am accepting this challenge too!

Starting out there is a problem with SQL Server in that the primary UI for manipulating the database and writing T-SQL (SSMS) don't really support the requirements for a computer game.

Following is a basic description (highly over-simplified) of the steps of a computer game:

Infinite Loop (aka Game Loop) that does following over and over and over:
  • User Input: Polls for user input (keyboard, mouse, joystick). User moves, fires gun, buys sword, saves girl, etc.
  • AI Actions: Makes "choices" using some kind of AI for non-player characters (NPCs) - movement, firing gun, etc.
  • Resolve Game World: Determines results of player and AI actions (collisions, hits/misses, increase score, affect of "gravity", decrease health of player/NPCs, etc).
  • Draw Game World: Computer redraws world and presents to player. Likely includes music/sounds.
SSMS and the SQL Server database engine support the required functionality of a computer game when examined individually but not when mashed together. SSMS doesn't support the polling/acceptance of user input when running a T-SQL script - so no Game Loop combined with the other 4 steps of the Game Loop in SSMS. There is a workaround if you want to "play" your game within SSMS - you as the user of SSMS can be responsible for the Game Loop and make the calls for User Input, AI Actions, Resolve Game World, and Draw Game World.

Let's get to the rules!
  1. Follows standard Tic-Tac-Toe rules. X goes first.
  2. Computer can either be Xs or Os. Set when you start new game.
  3. Game continues until one of the following occurs:
    • Player or computer wins.
    • No more open slots and no winner - this is a tie.
    • If you wish, you can short-circuit game and declare a tie if win is not possible. [Bonus points if you implement this.]
  4. Computer must always win if it can.
  5. Computer must tie if win not possible.
  6. Computer must never lose.
  7. Game "state" must be stored in the database (table, temporary table, TVP, variable, whatever).
  8. Input validation, in particular cheating prevention, must be done in the T-SQL.
  9. Determination of game board status must be down in database/T-SQL (that is, win state, tie state, game over state, etc.)
  10. Determination of computer's next move must be done in database/T-SQL.
If you wish you can implement the Game Looping, User Polling, World Redraw functionality in Powershell or in a .NET application or something similar but the core of the game must be a T-SQL game.

Post your setup and sample gameplay code here or on your own blog and a link here. Would love to see some collaboration going on!

Personal Aside: As I said before, I am accepting this challenge too. Following are some quick and dirty ideas that I might test out. I am not sure if the following ideas are a reasonable start of a solution or a dead end (likely a dead end!). T-SQL doesn't support arrays. I am not sure how I will represent the Tic-Tac-Toe board with nine slots (3 X 3). Maybe as 9 separate variables each representing a different slot on the board or maybe a 3 column 3 rows temporary table? But I think maybe a series of stored procedures to "play" a game. Maybe something like the following (somewhat mapped to the steps of the Game Loop)?
  • usp_StartGame(@ComputerPlays char(1), @GameID int OUTPUT) - This stored procedure starts a new game. The @ComputerPlays input is either 'X' or 'O' to set what the computer is playing. The OUTPUT parameter @GameID returns a new unique value representing the identifier for the game.
  • usp_ShowStatus(@GameID int) - This shows current state of the @GameID game. Redraws board possibly as formatted ASCII. Shows who is playing Xs and Os. Shows who's turn it is. Shows if Win/Loss/Tie/Game Over.
  • usp_PlayerTurn(@GameID int, @Position int) - Represents player move. @Position would represent one of the nine positions on the board. If the player attempted to cheat this would be caught here.
  • usp_ComputerTurn(@GameID int) - Represents computer move. This represents the meat of the coding. How do you code this so that the computer never loses. Hmmm. There are several ways to do this - recursively, brute force, data-driven, others? After the choice is made, this procedure possibly calls usp_ShowStatus to display board back to player showing computer's choice and results.
Please let me know if I need to add or clarify anything. Please let me know if you have any ideas or other feedback. Thanks for reading! Looking forward to seeing people's solutions and ideas!

Categories: SQL Server
Permalink | Comments (8) | Post RSSRSS comment feed

SQL Server Books Online As PDF Files

I would love to see Microsoft come out with the SQL Server Books Online (SQL BOL) as PDF files. The main reason is eBook readers like Amazon's Kindle. It would be great to have SQL BOL on the Kindle.

SQL BOL has been an excellent learning and reference tool for many years. I expect that SQL BOL will be continue to be my primary learning and reference resource in my career as a SQL Server professional. You can get access to SQL BOL several ways, including as part of a SQL Server installation, as part of the locally installed MSDN Library, as a separate downloadable installation file, and via the online MSDN Library or through TechNet both on Microsoft's website. But SQL BOL currently isn't available via PDF files. Arguably you could turn the HTML pages of SQL BOL into PDF files - but that would be a huge task. So getting SQL BOL onto your Kindle is a big challenge.

Let's contrast Microsoft's SQL Server documentation with Oracle's documentation. You can find Oracle's documentation here as both HTML and PDF files. You can download the Oracle PDF files either individually or as zipped collections. The Oracle PDF files play well with the Amazon Kindle. I'd love it if SQL BOL did as well!

I have created on the Microsoft SQL Server Connect website the following suggestion to make the SQL BOL available as PDF files. If you agree please up-vote on the Microsoft Connect website. If you have any ideas or feedback please share in the comments below. Thanks for reading!

Categories: SQL Server
Permalink | Comments (3) | Post RSSRSS comment feed

Strange INSTEAD OF Trigger Side Affect

In previous post here I described a seemingly strange side affect of an INSTEAD OF trigger. Within the INSTEAD OF trigger I didn't update any records - I just returned an error. No rows are actually being affected. However, SSMS (SQL Server Management Studio) shows records *are* being affected. The count is equal to the count that would have been affected had the INSTEAD OF trigger not existed or if it had been disabled.

I wanted to see if this was just SSMS behavior. I created a simple C# Console Application project in Visual Studio 2008 against a SQL Server 2008 Dev Ed database.

Following is database setup code.

CREATE DATABASE MyDB;
GO

USE MyDB
GO

CREATE TABLE dbo.MyTable
(
ID int IDENTITY(1, 1) NOT NULL PRIMARY KEY,
MyColumn varchar(50) NOT NULL
);
GO

-- sample data
INSERT INTO dbo.MyTable (MyColumn) VALUES ('One');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('Two');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('Three');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('Four');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('Five');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('Six');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('Seven');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('Eight');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('Nine');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('Ten');
GO

CREATE TRIGGER dbo.tr_MyTable_StopDataMods
ON dbo.MyTable
INSTEAD OF INSERT, DELETE, UPDATE
AS
BEGIN
  
  SET NOCOUNT ON;
  
  RAISERROR(
  N'The data in table %s is static. All data modification operations are blocked. Disable the trigger %s to modify table data.', -- Message text.
  16, -- Severity
  1, -- State,
  'dbo.MyTable', -- first argument (table name)
  'dbo.tr_MyTable_StopDataMods' -- second argument (trigger name)
  );
  
END
GO

Test with the following code.

UPDATE MyTable SET MyColumn = 'X';

Should see expected error message and the "10 row(s) affected" message. If you do a SELECT * FROM MyTable; you should see that the data was not modified.

Now let's switch gears from T-SQL over to C# and .NET. Following is complete code for the application. You may have to modify the Connection String as required. The database UPDATE operation is the same as above that you executed in SSMS that resulted in a error, no rows being updated, but SSMS showing "10 row(s) affected."

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data;
using System.Data.SqlClient;

namespace ConsoleApplication1
{
  class Program
  {
    static void Main(string[] args)
    {
      string strConnection = "Data Source=localhost;Initial Catalog=MyDB;Integrated Security=True";
      using (SqlConnection cn = new SqlConnection(strConnection))
      {
        cn.Open();

        using (SqlCommand cmd = new SqlCommand())
        {
          cmd.Connection = cn;
          cmd.CommandType = CommandType.Text;
          cmd.CommandText = "UPDATE MyTable SET MyColumn = 'X';";
          int RowsAffected = cmd.ExecuteNonQuery();
          Console.WriteLine("Rows Affected: " + RowsAffected.ToString());
        }
      }

      string x = Console.ReadLine(); // Keep Console from closing
    }
  }
}

Run the application in debug mode. Should error out on the cmd.ExecuteNonQuery command with an unhandled SqlException with the expected error message: "The data in table dbo.MyTable is static. All data modification operations are blocked. Disable the trigger dbo.tr_MyTable_StopDataMods to modify table data." See the screenshot below. In the locals window take a look at cmd.Non-Public members._rowsAffected. Shows a value of 10 (ten). Now isn't that interesting. This side affect associated with the INSTEAD OF trigger that updates zero rows is also seen when making calls through other APIs. To confirm this behavior you could possibly do additional testing using other INSERT, UPDATE, and DELETE commands.



Any feedback or ideas that you have will be appreciated. Thanks!

Update:

I modified the INSTEAD OF trigger and removed the RAISERROR command. Now the INSTEAD OF trigger just has the SET NOCOUNT ON command.

ALTER TRIGGER dbo.tr_MyTable_StopDataMods
ON dbo.MyTable
INSTEAD OF INSERT, DELETE, UPDATE
AS
BEGIN

SET NOCOUNT ON;

END
GO

When you rerun the C# Console app, it runs successfully without error and shows the following.



If you again look at the underlying data in the table - it is unaffected by the UPDATE statement. No rows were actually updated.

To further confirm disable the trigger and rerun the C# Console app. It should again show Rows Affected: 10. This time when you query the data in the table - the data will have been updated.

According to the MSDN documentation located here, the System.Data.SqlClient.SqlCommand.ExecuteNonQuery() method "Executes a Transact-SQL statement against the connection and returns the number of rows affected." The documentation also states "For UPDATE, INSERT, and DELETE statements, the return value is the number of rows affected by the command. When a trigger exists on a table being inserted or updated, the return value includes the number of rows affected by both the insert or update operation and the number of rows affected by the trigger or triggers. For all other types of statements, the return value is -1. If a rollback occurs, the return value is also -1." The documentation is not very helpful.

It is clear that when INSERT, UPDATE, or DELETE statements are executed against a table or view with an INSTEAD OF trigger, regardless of what code is actually executed within the INSTEAD OF trigger, the following is occuring: A SELECT statement is being run against the table or view to determine the number of rows that would have been affected had the trigger not existed. This value is being passed back to the process/application requesting the INSERT, UPDATE, or DELETE operation as the count of rows being affected even though the actual number of rows affected could be zero, the same, or something completely different.

Again, thanks for reading and any feedback is appreciated. Thanks!

Categories: SQL Server
Permalink | Comments (7) | Post RSSRSS comment feed

Simple Indexed View To Constrain Data

In a recent Twitter David Taylor (Blog|Twitter) asked for #sqlhelp on constraining data in a varchar column that was holding 2 different types of data. One type of data is textual and like '4 PK WITH NO SN'. The other type of data is 11-digit numbers. The constraint he wanted to add was he wanted to ensure the numbers were unique. Duplicates if the values were textual are ok. Following is a possible quick and dirty solution that I developed in SQL Server 2008 Dev Edition. I don't know if it will work for David and I am sure there are other possibly better options but the following appears to work ok.

Mark Vaillancourt (Blog|Twitter) wrote a similar post several months ago here.

Please share any thoughts or feedback you may have. Thanks!

IF (OBJECT_ID('dbo.vwMyTableNumericMyColumn', 'V') > 0)
  DROP VIEW dbo.vwMyTableNumericMyColumn;
IF (OBJECT_ID('dbo.MyTable', 'U') > 0)
  DROP TABLE dbo.MyTable;

CREATE TABLE dbo.MyTable
(
ID int IDENTITY(1, 1) NOT NULL PRIMARY KEY,
MyColumn varchar(50) NOT NULL
);
GO

-- sample data
INSERT INTO dbo.MyTable (MyColumn) VALUES ('11111111111');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('11111111112');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('11111111113');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('11111111114');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('11111111115');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('4 Pk With No SN');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('4 Pk With No SN');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('4 Pk With No SN');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('4 Pk With No SN');
INSERT INTO dbo.MyTable (MyColumn) VALUES ('4 Pk With No SN');
GO

-- create schemabound view
CREATE VIEW dbo.vwMyTableNumericMyColumn
WITH SCHEMABINDING
AS
SELECT ID, MyColumn
FROM dbo.MyTable
WHERE (ISNUMERIC(MyColumn) = 1);
GO

-- create unique clustered index on schemabound view
CREATE UNIQUE CLUSTERED INDEX vwMyTableNumericMyColumn ON dbo.vwMyTableNumericMyColumn
(
MyColumn ASC
);
GO

-- confirm data
SELECT * FROM dbo.MyTable;
SELECT * FROM dbo.vwMyTableNumericMyColumn;

-- will cause error
INSERT INTO dbo.MyTable (MyColumn) VALUES ('11111111111');
-- will cause error
UPDATE dbo.MyTable SET MyColumn = '11111111111' WHERE MyColumn = '11111111112'
-- will not cause error
INSERT INTO dbo.MyTable (MyColumn) VALUES ('4 Pk With No SN');
-- will not cause error
UPDATE dbo.MyTable SET MyColumn = '22222222222' WHERE MyColumn = '11111111112'

Categories: SQL Server
Permalink | Comments (2) | Post RSSRSS comment feed

Create INSERT Script for Existing Table Data Using the tablediff.exe Utility

You have a table named MyTable that contains X number of rows. You need a SQL script that performs the CREATE TABLE command and INSERTs the X rows. There are several ways to accomplish this - this blog post is just one way. SQL Server Managament Studio does a great job of generating a CREATE TABLE script but not the INSERT commands. Following is a quick procedure to generate the INSERT commands using the SQL Server command-line utility named tablediff.exe. In SQL Server 2008 this application is located here - assuming default installation (C:\Program Files\Microsoft SQL Server\100\COM).

The tablediff.exe utility is normally used for replication synching but can be repurposed to generate INSERT statements.

First, create an empty copy of MyTable.

SELECT *
INTO dbo.MyTableCopy
FROM dbo.MyTable
WHERE 0 = 1;

Next execute the following command from a Windows command-line. You will need to change directories to where the tablediff utility is located first.

C:\Program Files\Microsoft SQL Server\100\COM>tablediff.exe -sourceserver MyServer -sourcedatabase MyDatabase -sourcetable MyTable -sourceuser sa -sourcepassword sa -destinationserver MyServer -destinationdatabase MyDatabase -destinationtable MyTableCopy -destinationuser sa -destinationpassword sa -f c:\temp\insert_MyTable.sql

Please note that rather than using named SQL Server logins you can use Windows Authentication and exclude the user/password values. Also the C:\temp directory may need to be created if it doesn't already exist. The process fails if the insert_MyTable.sql file already exists.

The insert_MyTable.sql file will contain the INSERT statements for the X number of rows in the original MyTable table. You will have to open the file in a text editor and global replace MyTableCopy with MyTable. If you combine this INSERT command list with the CREATE TABLE script from SSMS you have what you need to recreate the table and the data contained therein.

For more information about the tablediff.exe utility either go to SQL BOL or type in the following at the Windows command-line.

tablediff.exe /?

Please share any feedback that you may have. Thanks!

Categories: SQL Server
Permalink | Comments (1) | Post RSSRSS comment feed

Fiscal Year Implementation in a Database

What is a Fiscal Year? Also known as a financial or tax year, it is an accounting time period of 12-months that may or may not coincide with the calendar year. A fiscal year is the time period used to group various financial statements together for a given company or organization to create an annual or yearly report. Some companies have fiscal years that match the calendar year, but this is not the norm. Most companies and organizations have non-calendar year fiscal years that begin and end on the same date every year. The US Government has a fiscal year that runs from October 1st to September 30th. Some companies have fiscal years that last 52 weeks so that the fiscal year starts on the same day of the week every year - in affect 2 days less than an actual year. Cisco is an example of a company that has a 52 week fiscal year. Another characteristic of fiscal years is that they sometime change when companies undergo mergers, buyouts, or restructuring.

This blog post represents a possible solution for implementing Fiscal Years in your database. For most companies and organizations it would likely be ok to hardcode fiscal year information/calculations within user-defined functions or within other code blocks, however this solution is more generic, data driven and flexible.

The foundation for the solution is a simple table named FiscalYear (note: code tested on Microsoft SQL Server 2008 Dev Ed):

CREATE TABLE FiscalYear
(FiscalYear int NOT NULL,
BeginDate date NOT NULL,
EndDate date NOT NULL,
CONSTRAINT PK_FiscalYear_FiscalYear PRIMARY KEY CLUSTERED (FiscalYear ASC))

Add the following CHECK CONSTRAINTS to the FiscalYear table to ensure quality of data. The first requires BeginDate to be before EndDate. The second requires that FiscalYear value match the Year(EndDate) value - this is the norm, the year of the end date defines the fiscal year.

ALTER TABLE dbo.FiscalYear ADD CONSTRAINT CK_FiscalYear_BeginDate_lt_EndDate CHECK (BeginDate < EndDate);
ALTER TABLE dbo.FiscalYear ADD CONSTRAINT CK_FiscalYear_FiscalYear_equals_EndDateYear CHECK (FiscalYear = YEAR(EndDate));

Next populate the new FiscalYear table with some sample data:

INSERT FiscalYear (FiscalYear, BeginDate, EndDate) VALUES (2007, '20061001', '20070930');
INSERT FiscalYear (FiscalYear, BeginDate, EndDate) VALUES (2008, '20071001', '20080930');
INSERT FiscalYear (FiscalYear, BeginDate, EndDate) VALUES (2009, '20081001', '20090930');
INSERT FiscalYear (FiscalYear, BeginDate, EndDate) VALUES (2010, '20091001', '20100930');
INSERT FiscalYear (FiscalYear, BeginDate, EndDate) VALUES (2011, '20101001', '20110930');

Please note that valid data for this table requires that there be no overlaps or gaps in the FiscalYear values and BeginDate/EndDate values. Later in this post I will provide an solution to the intra-table data validation problem.

Next we look at querying the data contained within the table. We want to encapsulate the following code within a user-defined function so that it behaves very similarly to the Transact-SQL YEAR scalar function:

DECLARE @InputDate date;
SET @InputDate = '20091201';
SELECT fy.FiscalYear FROM FiscalYear fy WHERE @InputDate BETWEEN fy.BeginDate AND fy.EndDate;

Looking at the Execution Plan of the above SELECT query a Clustered Index Scan is used - an Index Seek is preferred. Add the following index to speed performance.

CREATE NONCLUSTERED INDEX IX_FiscalYear_BeginDate_EndDate_FiscalYear ON FiscalYear
(BeginDate ASC, EndDate ASC, FiscalYear ASC)

The SELECT query above should now used an Index Seek (Non-Clustered) which should improve performance. Now convert the SELECT query into a user-defined scalar-valued function.

CREATE FUNCTION dbo.udfFiscalYear(@InputDate date = NULL)
RETURNS int
AS
BEGIN
  DECLARE @Return int;

  IF (@InputDate IS NULL)
  BEGIN
    SET @InputDate = GETDATE();
  END

  SELECT @Return = fy.FiscalYear FROM dbo.FiscalYear fy WHERE @InputDate BETWEEN fy.BeginDate AND fy.EndDate;

  RETURN @Return;
END

If a NULL value is passed into the dbo.udfFiscalYear function, it is assumed that you are requesting the current fiscal year. In this case, the @InputDate value is converted to GETDATE() before the FiscalYear table lookup is done. Also, if no matching row in the FiscalYear table is found, the function returns NULL.

We now have the basic structures (lookup table and user-defined function) in place to implement fiscal years within your database. In subsequent post I will describe specific uses of this fiscal year implementation.

Finally, in this post I will supply a solution to the problem of ensuring there are no gaps or overlaps in the data contained within the FiscalYear table. This solution is not ideal because it is not a real-time solution but rather after-the-fact. Potentially bad data could be entered into the table but not caught and hopefully fixed until after the following variance query below is ran. The query relies on two user-defined functions that are listed.

CREATE FUNCTION dbo.udfGetPreviousFiscalYearEndDate(@InputFiscalYear int)
RETURNS date
AS
BEGIN
  DECLARE @Return date;

  SELECT @Return = fy.EndDate FROM dbo.FiscalYear fy WHERE fy.FiscalYear = (@InputFiscalYear - 1);

  RETURN @Return;
END

and

CREATE FUNCTION dbo.udfGetNextFiscalYearBeginDate(@InputFiscalYear int)
RETURNS date
AS
BEGIN
  DECLARE @Return date;

  SELECT @Return = fy.BeginDate FROM dbo.FiscalYear fy WHERE fy.FiscalYear = (@InputFiscalYear + 1);

  RETURN @Return;
END

The following query should show 1s (ones) across the board for PreviousEndDateDiff and NextBeginDateDiff (except for first fiscal year which will show NULL for PreviousEndDateDiff and last fiscal year which will show NULL for NextBeginDateDiff).

SELECT fy.FiscalYear, fy.BeginDate, fy.EndDate,
DATEDIFF(d, dbo.udfGetPreviousFiscalYearEndDate(fy.FiscalYear), fy.BeginDate) PreviousEndDateDiff,
DATEDIFF(d, fy.EndDate, dbo.udfGetNextFiscalYearBeginDate(fy.FiscalYear)) NextBeginDateDiff
FROM FiscalYear fy
ORDER BY fy.FiscalYear

If 1s (ones) are not shown then you have problems with the FiscalYear table data that needs to be fixed.

In closing, in a future post I will explore better solutions to the data validation problem possibly using CHECK CONSTRAINTS, triggers, stored procedures or other mechanisms. I strongly prefer a real-time data validation mechanism over an after-the-fact find and cleanup process. Also, as stated before I will write further about using this fiscal year implementation in a practical way in your database.

If you have any ideas, criticisms, or questions please let me know. Any feedback is appreciated! Thanks!

Categories: SQL Server
Permalink | Comments (0) | Post RSSRSS comment feed

Stopping SQL Injections Attacks

In the previous post here I did the following three things:
  1. Described what a SQL Injection Attack is.
  2. Provided a simple example of a SQL Injection-Susceptible Web Site built using ASP.NET, C#, ADO.NET, SQL Server, and the AdventureWorks sample database.
  3. Provided some sample scripts to "Hack" or perform SQL Injection Attacks on the Web Site that we built.
From the results of the previous post you can see that if you are part of a company with a web site on the public Internet and that web site is susceptible to SQL Injection attacks, you and your company and your customers could be in a world of some serious hurt.

I had some fun putting together the previous example. It was kinda cool to see what all I could do with the database and database server from just a simple website with a textbox, button, and grid. I can sorta understand the rush that a hacker might get from the process of figuring out about a database and database server from afar - Ooh! Look at that! Look what I can do! But this is serious business. In today's world, data stored in databases and the movement of that data across the Internet is the world's lifeblood. I don't carry cash any more. I carry a debit card. All of my money is a mishmash of ones and zeros stored in some database on some server at some bank. I go online to check my balance. I swipe my card at the grocery store when I buy a gallon of milk. Is it all a house of cards? Is there a hacker out there playing the role of Tyler Durden about to bring the system all tumbling down? What can I as a software developer, a database developer, a Microsoft SQL Server Database Developer do to help protect this system and help build solid, robust, and secure data recording, reporting, and workflow applications and the associated databases such that my employer succeeds? What can I do to help raise my own level of professionalism and work quality, along with possibly helping other database professionals along the way?

Was the previous paragraph too Jerry Maguire-y?

To help fulfill my ambition of building better software and helping stop problems like SQL Injection, I am going to share what I know. I am sure that I have built SQL Injection-Susceptible Web Sites in the past - I won't in the future. My hope is that someone doesn't make the same mistakes that I did. My hope is that it might help someone at some point. I hope other SQL Server professionals and other developers jump in and provide their ideas too!

In this post I am going to provide concrete ways of stopping SQL Injection Attacks. But first I want to clarify things. My focus in on Microsoft technologies so I used Microsoft technologies for my example SQL Injection-Susceptible Web Site. SQL Injections are NOT just a problem for the Microsoft world. All Server-Side Web Development programming languages such as PHP, Perl, Ruby, and C# are susceptible to SQL Injections attacks. All of the various data access APIs such as the PHP:MySQL and ADO.NET libraries are susceptible. All SQL-based databases engines including MySQL, Oracle, and SQL Server are susceptible.

Hackers perform SQL Injection Attacks. They are ultimately responsible for the problems, security violations, unauthorized accesses, loss of data, damaged servers, and loss of creditibility and trust of all business software developers, database professionals, server administrators, and networking gurus. That being said, I believe that on the software developer, database, server, and networking side, the source of the SQL Injection problem is Application Developers. The DBA can mitigate the problem by restricting the App Devs to using only stored procedures which most App Devs will fight tooth and nail, plus the DBA can be ultra-restrictive with the login used for database access which App Devs will fight too - "just make the login dbo (database owner) for simplicity's sake." But ultimately, DBAs cannot do anything practical to stop SQL Injection Attacks (other than shutdown the database server). Ultimately, App Devs are responsible for building a SQL Injection-Susceptible Web Site. If SQL is inline or embedded with their code and string concatenation is used to build SQL statements (INSERTs, UPDATEs, DELETEs, and SELECTs) then you have the strong potential for having a Web Site that is SQL Injection-Susceptible.

Avoid SQL creation statements similar to the following. Avoid SQL built with string concatenation.

cmd.CommandText = "SELECT ProductID, ProductNumber, Name, ListPrice FROM Production.Product WHERE Name LIKE '%" + this.TextBox1.Text + "%'";

Rather than using string concatenation to build your SQL statements, use Parameterized Queries. Parameterized Queries can provide strong type checking (including size) and input isolation (where multiple commands chained together with the command separator (semicolon) are not possible). To see this add another TextBox and Button to the Default.aspx page of the previous ASP.NET/C# Web Application and add the following code to the Click event of Button2.

protected void Button2_Click(object sender, EventArgs e)
{
  string strConnection = "Data Source=SERVER;Initial Catalog=AdventureWorks;User Id=sa;Password=sa;";
  using (SqlConnection cn = new SqlConnection(strConnection))
  {
    cn.Open();

    using (SqlCommand cmd = new SqlCommand())
    {
      cmd.Connection = cn;
      cmd.CommandType = CommandType.Text;
      cmd.CommandText = "SELECT ProductID, ProductNumber, Name, ListPrice FROM Production.Product WHERE Name LIKE '%' + @search + '%';";
      cmd.Parameters.AddWithValue("@search", this.TextBox2.Text);

      using (SqlDataReader dr = cmd.ExecuteReader())
      {
        this.GridView1.DataSource = dr;
        this.GridView1.DataBind();
      }
    }
  }
}

The differences between the Button1_Click and the Button2_Click appear to be minor. I stated that you needed to avoid string concatenation to stop SQL Injection. That is true. [Clarification From The Idiot-Writer Time!] The LIKE command in SQL is a bit problematic and to get it to work with the % (Percent) Wildcard or Any Characters Wildcard you still have to concatenate to build your SQL string. But the Button2_Click code or the Parameterized Query code uses a named parameter or placeholder (@search) that is populated in the subsequent statement - "cmd.Parameters.AddWithValue("@search", this.TextBox2.Text);". This is the key step to stopping SQL Injection attacks. Rather than saying Avoid String Contenation When Building Your SQL I should say Use Parameterized Queries To Build You SQL.

Go back and attempt to "run" the same hacker scripts in this second TextBox using the second Button. Nothing appears to happen. What is happening is that you are now effectively searching for an item in the product table with a name that matches the big long string that you entered into the TextBox. You have stopped the SQL Injection Attack.

In doing some testing I found something - it is likely others have already pointed it out before me. SQL Profiler and possibly the system views can be used by the DBA to find possible SQL Injection risks. If you run and observe the SELECT queries from Button1_Click and Button2_Click you see that both run as the ApplicationName ".Net SqlClient Data Provider". Button1_Click runs the query as "straight" SQL. Button2_Click runs using the sp_executesql stored procedure. This could possibly be leveraged to find ADO.NET applications that are not using Parameterized Queries and could possibly be SQL Injection-Susceptible. But this is not an absolute test for sure. This idea could possibly be extended to other SQL Server access libraries and other applications. Plus possibly used with other databases such as MySQL and Oracle - I don't know.

The use of Parameterized Queries over what I will call manually built SQL in App Dev code is the key to preventing SQL Injection Attacks.

Following are some additional things beyond the use of Parameterized Queries to help stop SQL Injection Attack (I will avoid the mountains of narrative on these items! <grin> ):
  • Server-side data validation of all inputs before touching database. Never trust client-side validation exclusively.
  • Use an ORM such as LINQ to SQL, Entity Framework, or nHibernate to cross the application code/SQL code boundary.
  • Possibly use Stored Procedures for all database access. No direct SELECTs, INSERTs, UPDATEs, and DELETEs.
  • Separate your web application code from the data access layer code by using web services. Ensure inputs are heavily validated on the web services.
  • GRANT only what is absolutely required to the web application's login. Never use a dbo (database owner) and definitely not a sysadmin login for an application's login. DENY access to everything not absolutely required. Follow the best practice of least priviledge.
  • If the web application is an ASP.NET consider using a specialized logging tool such as ELMAH. Can be added on to existing compiled sites very easily and provide view of all errors including notifications. There are commercial equivalents along with equivalents for other platforms also. You can also roll your own error logging and notification system to help keep an eye on SQL Injection Attacks and other penetration attempts.
There are likely several other methods to stopping or helping mitigate the threat of SQL Injection Attacks. But the key is getting Web Application Developers to use Parameterized Queries if they are going to be making direct SQL calls against the database.

If anyone has any additional ideas please pass them along. Any feedback is appreciated. Thanks!

Categories: SQL Server
Permalink | Comments (0) | Post RSSRSS comment feed