Showing posts with label sql. Show all posts
Showing posts with label sql. Show all posts

Tuesday, 22 November 2016

Connection IP’s to my database

Thanks to @j_mcmullan

Apperently there's a lot more info on these tables

 select distinct--c.session_id
	  c.client_net_address
	  ,s.database_id
	  ,db_name( s.database_id)
from sys.dm_exec_connections c
    inner join sys.dm_exec_sessions s on c.session_id = s.session_id
where s.database_id = db_id('ASPState')
GO

(I beleiveit may be ironical that I was looking at sessions to a session database).

Friday, 18 November 2016

Sprints 2 and 3 retrospective–an hour really isn’t long.

So – sprint 2 took about an hour and a half of coding – so a lot closer than the first.

I then noticed that I still had some inline nasty SQL in my powershell – so I cracked out another hour and got another proc and changes in.

Main lesson so far – an hour is a really short time – keep pebbleising the tasks.

Outcome

Ok – so the work got done and hopefully I’ll have a list of riders on production soon.

 

I’ve thought hard about it and its definitely a useful task towards the end goal as the first “Minimum” viable product is for riders to sign up.  I’m going to allow google authentication/Oath which will strongly (enough for this) validate the users email which I can then use to look up there details in the data I have.

 

 

A good start.

 

Some feedback on the tasks here – really putting this code here for anyone to look @ until I work out if/what to open source.  Unless you particularly want to critique my sql or look at how I use powerhell to call stored procs then not much excitement.

 

“Work out how to deploy database project to local database and how to connect to it. “

this was pretty easy – just set the PowerShell script to depend on the database project and then it deploys all your changes as it builds – pretty nifty stuff

 

“Create stored procs for crud operations required by script – include find rider.”

CREATE PROCEDURE [dbo].[ups_RiderInsert]
    @vcrFirstName    NVARCHAR (100) ,
    @vcrSurname      NVARCHAR (100) ,
    @vcrAddressLine1 NVARCHAR (100) ,
    @vcrAddressLine2 NVARCHAR (100) ,
    @vcrTown         VARCHAR (100)  ,
    @vcrCountryCode  VARCHAR (2)    ,
    @vcrCountry      VARCHAR (100)  ,
    @vcrPhoneNumber  VARCHAR (20)   ,
    @vcrClub         VARCHAR (50)   ,
    @vcrClubNumber   VARCHAR (20)   ,
    @vcrECName       VARCHAR (100)  ,
    @vcrECTelephone  VARCHAR (100) 
AS
SET XACT_ABORT ON

BEGIN TRANSACTION
	INSERT INTO tblRider
	VALUES( 
	@vcrFirstName,
	 @vcrSurname      ,
    @vcrAddressLine1 ,
    @vcrAddressLine2 ,
    @vcrTown           ,
    @vcrCountryCode  ,
    @vcrCountry        ,
    @vcrPhoneNumber  ,
    @vcrClub            ,
    @vcrClubNumber      ,
    @vcrECName       ,
    @vcrECTelephone)

	SELECT SCOPE_IDENTITY()  AS INT
COMMIT TRANSACTION



CREATE PROCEDURE [dbo].[usp_EventParticipentInsert]
    @intRiderId       INT  ,
    @intEventId       INT  ,
    @decPaidSTG       MONEY,
    @decPaidEUR       MONEY
AS
SET XACT_ABORT ON
BEGIN TRANSACTION
INSERT INTO tblEventParticipant
VALUES(
   @intRiderId       ,
    @intEventId      ,
    @decPaidSTG      ,
    @decPaidEUR      
)
	SELECT SCOPE_IDENTITY()  AS INT

COMMIT
CREATE PROCEDURE [dbo].[usp_LicenceGetByRiderAndYear]
	@intRiderId int,
	@intYear int
AS
	SELECT * FROM dbo.tblLicense 
	WHERE
	intRiderId = @intRiderId
	AND intYear = @intYear
CREATE PROCEDURE [dbo].[usp_LicenseInsert]
    @intYear          INT,
    @intRiderId       INT,
    @vcrLicenceNumber NVARCHAR(20)

AS
SET XACT_ABORT ON
BEGIN TRANSACTION
INSERT INTO tblLicense 
VALUES(@intYear , @intRiderId, @vcrLicenceNumber)
COMMIT
CREATE PROCEDURE [dbo].[usp_RiderSearchByName]
	@FirstName nvarchar(100) ,
	@SurName nvarchar(100)
AS
	SELECT * FROM tblRider r
	WHERE (r.vcrFirstName like @FirstName + '%' OR @FirstName IS NULL)
	AND (r.vcrSurname like @SurName + '%' OR @SurName IS NULL)
	

 

 

    “Enahance script / tidy up for stored proc use and other comments in script below.”

param (
	   [Parameter(Mandatory=$true)][string]$fileName,
		[Parameter(Mandatory=$true)][string]$connString,
		[Parameter(Mandatory=$true)][int]$intEventId,
	[Parameter(Mandatory=$true)][int]$intYear
)


function ToDBNull($var) {
	if($var -eq $null) {return [System.DBNull]::Value}
	return $var;
}

#
# Script.ps1
# TODO - rename the bit above and document this method
function parseExcel([String] $fileName){
	$excel=new-object -com excel.application
	$wb=$excel.workbooks.open($fileName)
	$sh = $wb.Sheets.Item(1)
	# TODO de hard code teh endrow.
	$startrow = 2
	$endrow = 16#Connection and Query Info

# prpbably best to pass the database name etc. through as Parameters

$conn = new-object System.Data.SqlClient.SqlConnection $connString 
	#TODO Should the open be in the try/catch?
$conn.Open()
try{

	# Main for loop for parsing sheet
	# TODO rather than hard coding rows do until no end.

	for($i = $startrow;
		($sh.Cells.Item($i,1).Value2 -ne $null) -and ($sh.Cells.Item($i,1).Value2.Trim() -ne "") ; 
		$i++) {
$Name = $sh.Cells.Item($i,1).Value2
$Surname = $sh.Cells.Item($i,2).Value2
[String]$Address = $sh.Cells.Item($i,3).Value2
$Tel = $sh.Cells.Item($i,4).Value2
$email = $sh.Cells.Item($i,5).Value2
$Club = $sh.Cells.Item($i,6).Value2
$CI = $sh.Cells.Item($i,7).Value2
$ClubNo = $sh.Cells.Item($i,8).Value2();
$EC = $sh.Cells.Item($i,9).Value2
$ECTel = $sh.Cells.Item($i,10).Value2
$PaidE = $sh.Cells.Item($i,11).Value2
$PaidS = $sh.Cells.Item($i,12).Value2 
		# TODO - do we need to close the command - check documentation 
		# to see if it implements IDisposable - and is there a using equivalent in PowerShell
[System.Data.SQLClient.SQLCommand]$Command = New-Object System.Data.SQLClient.SQLCommand

	$Command.Connection = $conn
			$Command.CommandType = [System.Data.CommandType]::StoredProcedure
			# - lookup first - do within proc?.

	$Command.CommandText = "[dbo].[usp_RiderSearchByName]"
	
	$Command.Parameters.aDD("@FirstName",[System.Data.SqlDbType]::NVarChar)
	$Command.Parameters["@FirstName"].Value = $Name
	$Command.Parameters.Add("@SurName",[System.Data.SqlDbType]::NVarChar ).Value = $Surname
	[int]$ID = $null
   [System.Data.SqlClient.SqlDataReader ]$reader = $Command.ExecuteReader()
			try{
				if($reader.Read()){
					$ID = $reader["intRiderID"]
				}
			}
			finally{
				$reader.Close()
			}
			# if we didn'te find a rider then create on
			if($ID -eq 0) {
				[System.Data.SQLClient.SQLCommand]$Command =  $Command = New-Object System.Data.SQLClient.SQLCommand

			$Command.Connection = $conn
			$command.CommandType = [System.Data.CommandType]::StoredProcedure

				$Command.CommandText = "[dbo].[ups_RiderInsert]"
				$Command.Parameters.Add("@vcrFirstName",[System.Data.SqlDbType]::NVarChar ).Value = $Name
				$Command.Parameters.Add("@vcrSurname",[System.Data.SqlDbType]::NVarChar ).Value = $Surname
				$Command.Parameters.Add("@vcrAddressLine1",[System.Data.SqlDbType]::NVarChar ).Value = ToDBNull($Address.Split(',')[0])
				$Command.Parameters.Add("@vcrAddressLine2",[System.Data.SqlDbType]::NVarChar ).Value = ToDBNull($Address.Split(',')[1])
				$Command.Parameters.Add("@vcrTown",[System.Data.SqlDbType]::NVarChar ).Value = ToDBNull($Address.Split(',')[2])
				$Command.Parameters.Add("@vcrCountryCode",[System.Data.SqlDbType]::NVarChar ).Value = ToDBNull($null)
				$Command.Parameters.Add("@vcrCountry",[System.Data.SqlDbType]::NVarChar ).Value = ToDBNull($null)
				$Command.Parameters.Add("@vcrPhoneNumber",[System.Data.SqlDbType]::NVarChar ).Value = ToDBNull($Tel)
				$Command.Parameters.Add("@vcrClub",[System.Data.SqlDbType]::NVarChar ).Value = ToDBNull($Club)
				$Command.Parameters.Add("@vcrClubNumber",[System.Data.SqlDbType]::NVarChar ).Value = ToDBNull($ClubNo)
				$Command.Parameters.Add("@vcrECName",[System.Data.SqlDbType]::NVarChar ).Value = ToDBNull($EC)
				$Command.Parameters.Add("@vcrECTelephone",[System.Data.SqlDbType]::NVarChar ).Value = ToDBNull($ECTel)
		
				[int]$ID = $Command.ExecuteScalar()
				}
	
[System.Data.SQLClient.SQLCommand]$Command =  $Command = New-Object System.Data.SQLClient.SQLCommand

	$Command.Connection = $conn
			$command.CommandType = [System.Data.CommandType]::StoredProcedure
	$Command.CommandText = "[dbo].[usp_EventParticipentInsert]"
				$Command.Parameters.Add("@intRiderId",[System.Data.SqlDbType]::Int).Value = $ID
				$Command.Parameters.Add("@intEventId",[System.Data.SqlDbType]::Int).Value = $intEventId
					$Command.Parameters.Add("@decPaidSTG",[System.Data.SqlDbType]::Money).Value = If($PaidS -ne $null)  {$PaidS} else {0}
					$Command.Parameters.Add("@decPaidEUR",[System.Data.SqlDbType]::Money).Value = If($PaidE -ne $null)  {$PaidE} else {0}


		

	$Command.ExecuteScalar()
			# if got a CI number
			if($CI -ne $null -and $CI.Trim() -ne '') { 
# search for a licence

			[System.Data.SQLClient.SQLCommand]$Command =  $Command = New-Object System.Data.SQLClient.SQLCommand

			$Command.Connection = $conn
			$command.CommandType = [System.Data.CommandType]::StoredProcedure
			$Command.CommandText = "[dbo].[usp_LicenceGetByRiderAndYear]"
			$Command.Parameters.Add("@intRiderId",[System.Data.SqlDbType]::Int).Value = $ID
			$Command.Parameters.Add("@intYear",[System.Data.SqlDbType]::Int).Value = $intYear
	
			[int]$LID = $null
   [System.Data.SqlClient.SqlDataReader ]$reader = $Command.ExecuteReader()
			try{
				if($reader.Read()){
					$LID = $reader["intRiderID"]
				}
			}
			finally{
				$reader.Close()
			}
				# no licence found - create one
	if($LID -eq 0){
	[System.Data.SQLClient.SQLCommand]$Command =  $Command = New-Object System.Data.SQLClient.SQLCommand

			$Command.Connection = $conn
			$command.CommandType = [System.Data.CommandType]::StoredProcedure
			$Command.CommandText = "[dbo].[usp_LicenseInsert]"
			$Command.Parameters.Add("@intRiderId",[System.Data.SqlDbType]::Int).Value = $ID
			$Command.Parameters.Add("@intYear",[System.Data.SqlDbType]::Int).Value = $intYear
	$Command.Parameters.Add("@vcrLicenceNumber",[System.Data.SqlDbType]::NVarChar).Value = $CI
		$Command.ExecuteScalar()
				}
	

	}

	}
}
Finally
{
$conn.Close()
}
$wb.Close()
$excel.Quit()
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($excel)

	}


	parseExcel($fileName)

Sunday, 30 October 2016

Sprint One - As "Stuart McLean" I want to be able to get a csv list of entrants for a brevet.

The physc evals tell me I'm an implementer - which means I get cracking on things but, as many around me know, rarely finish.  You need a completer finisher for that.
So, I thought a lot and put one thing on the backlog that I might be able to achieve in an hour.

Breaking down the user story

It looks pretty innocuous - right - but - believe it or not, there has been a fair amount of thought into such a simple story.

Let's start with the actor.


"Stuart McLean" - why not a brevet organiser.


Well - there is a whole bunch of stuff that needs doing do secure the average online applicaiton - authentication, authorisation, deployment, hosting etc.  As "Stuart McLean" I can circumnavigate all of these and focus on the data!

Get a list of csv entrants

So as I said in my last blog - I already have a way of using excel to do lots of the admin.  The big task is to get the data in a spreadsheet.

For a brevet

Well - I guess it's really for the next brevet!

What I have

Well - I guess I already have a list of previous people and brevets.
I also know what brevets are coming up.

Solution Overview

I'm going to create a database - whoop whoop - to store:
Riders,
The Brevets,
Entrants to a brevet - i.e. the relationship between the Riders and the Brevets.
Then I'm going to upload the data from the excel files I have.  This should give me a list of riders, existing entries and previous brevets.
This should validate my data model.
I'm going to do all this in SQL Server management studio straight on the database.

Can I Sprint in an hour?

ACP Brevet Cards

It seems the more I know, the less I get to do.  I'm currently a technical architect for 6 sprint teams - which is exciting and challenging but I'm getting less and less (read no) time at the coal face.
The pace of change in IT seems to be exponential and I like to keep my toes on the edge of the programming curve.
OK - so I'd love to get back into Unix, c++, java and all that stuff - but it's been 15 years or so so I'm going to stick with the Microsoft stack.

The Challenge

Deliver a user story in one hour.  That's one hour to code, test and release.  
"Give me six hours to chop down a tree and I will spend the first four sharpening the axe." 
Abraham Lincoln

I am going to allow - outside of the hour - study and plan.  Hopefully this will make the hour super productive.

The project

The project is simple - and combines programming with my other passion - cycling.  You can read about some of my cycling here.  Every year I organise a few "Brevets" - long distance cycle rides for Audax Ireland.  There is a bit of admin involved:

  1. Collect details of participants - currently they email or post a form with name, address etc.
  2. Collect money - I accept PayPal or cash on the day.  Not all organisers accept cash and some push the PayPal through the clubs.  Different events cost different amounts.  Payment can be made in € or £.
  3. Print "Brevet Cards" with the event details, riders details and control points.
  4. Sign on sheet for Cycling Ireland (CI) with list of riders to collect signature at start.
  5. One day licence form for riders who aren't members of cycling Ireland.
  6. On completion - fill in finisher details on spreadsheet for Audax Club Parisien to homogolate the rides and order medals where required.
  7. Submit accounts and pay balance toe Audax Ireland (AI).
  8. When homogolation stickers and medals arrive, post with brevet cards to riders.
Currently I run all this using a spreadsheet to collect the rider info on.  I then have a word mail merge for the Brevet cards and cut and paste onto the other documents.
The biggest pain is actually getting the data on the spreadsheet.  It is either keyed or cut an pasted depending on how the application arrives.  I can also sometimes cut and paste from an old spreadsheet - but the CI license numbers change every year and other details may change.

The tech

  • SQL Azure - assuming a relational database - might need other azure tech around the outside for large data or whatever.
  • C#.net for business object layer, processes etc.
  • MVC for serving HTML  to client
  • WEB API - for services
  • Angular
  • Visual Studio

Entity framework or ADO.NET + Strongly typed stored procedures.

Its a  question that's been worrying me for about 8 years!  
Currently I have projects working with ADO.NET and stored procs (most), EF and unit of work, and EF over stored procedures.  I'm still torn between development speed, security, maintanence and 

Strongly Typed Stored Procedures

  • Tick a big security box - giving lease privilege access to the database and strong type checking - a string is a string never a piece of sql, thus adding a layer of defence against many forms of attack, SQL injection being the most obvious.  OK - O know EF will have vendor backing and is paramaterised etc. etc. but believe me, when some external auditor comes in and says what layers of defence do you have against attack, the strongly typed stored procedure with least privilege (i.e. connection can only execute stored procs) is king.
  •  Are fast
  • Do set based operations where they belong - in an engine that has been optimised over about twenty five years for performing set based operations!
  • Allow provide an encapsulation layer over your data model, allowing it to be changed and maintained independent(ish) of your code.
  • Allow your database to be normalised and your object model to be de-normalised as. it should be.
  • Allow your interactions with the database to be as they should i.e. full control over how much data is fetched when.
  • Is stable with a very long support life ahead and behond - ADO.NET.
  • Can lead to business logic being scattered across code and sql depending on who wrote it!
  • Can result in stored procedure mountains.
  • Offer a great maintainence fix.

Entity Framework

  • Is quicker to develop - though I'd argue this might only be over the first few iterations of a project.
  • Supports more flexible query functionality for the application e.g. sorting, querying multiple parameters etc. without additional effort.
  • Can leave you fighting the framework.
  • Is not as stable
  • Is not as secure
  • Through web services - offers a consistent business layer for other applications e.g. reporting etc.
  • May lead to compromises in the data model (de-normalised) or the business object model (too normalised).
  • Requires a full code deploy.
I'm going to put this decision off a bit longer!

Project management and tools

OK - so its agile, obviously.  

Agile tool/work tracking

TBD - I have visual studio online/visual studio team system and this is the obvious choice but I might look at a more "open" product - after all I'm wearing shorts right!
I guess initially it'll be this blog as my Minimum Viable Product in an hour is going to be very minimum.

Source Code Repository

To open source or not to open source, that is the question - well - I really want to open source - not that I'm expecting a million IT pros to rock up and start working on this but, like I say, I wearing shorts.  I thing I need to be careful of is ideally I'd use some of Stiona Software's utility and platform code but this has IPR wrapped in it and I'm not wearing sandals.  Software is my living too.

Again - with my shorts on GIT - but frankly it's something else to learn at the moment.

So - probably codeplex.  I have an account and it integrates with visual studio.

Monday, 19 October 2015

How to specify dates in sql scripts.

Always use the ISO 8601 date format

yyyy-mm-ddThh:mm:ss[.mmm]
You must specify everything except the [.mmm] e.g. 2014-12-24T00:00:00
 

How I create history or audit tables.

OK – not too complex this but quick and easy.

 

Firstly I create a history table – to do this I get SQL management studio to generate a create script for the table I want to audit.  I add the postscript “History” to the table name, remove the constraints and indexes and add in 2 extra columns, one for the type of update and one for the date/time e.g.

 

CREATE TABLE [dbo].[tblProductCodeHistory]
(
    chrChangeType nchar(1) NOT NULL,
    dteChangeTime DateTime NOT NULL,
    [intProductId] [int]  NOT NULL,
    [intSalesAccountId] [int] NOT NULL,
    [intSalesTaxCodeId] [int] NOT NULL,
    [vcrDescription] [nvarchar](100) NULL,
    [intBusinessId] [int] NOT NULL,
    [decNetPrice] [money] NULL,
    [vcrProductCode] [nvarchar](100) NOT NULL,
    [decCost] [money] NULL,
    [intSupplierAccountId] [int] NULL,
    [intProductTypeId] [int] NOT NULL,
    [intQuantity] [decimal](10, 3) NOT NULL ,
    [intUserId] [int] NULL,
    [dteUpdated] [datetime] NULL,
    [vcrUnit] [dbo].[ShortString] NULL DEFAULT (NULL),
) ON [PRIMARY]
GO

Then I create a couple of triggers on the original table as follows -

CREATE TRIGGER [dbo].[ProductCode_Delete]
   ON  [dbo].[tblProductCode]
   AFTER DELETE
AS
BEGIN
    -- SET NOCOUNT ON added to prevent extra result sets from
    -- interfering with SELECT statements.
    SET NOCOUNT ON;
    INSERT INTO tblProductCodeHistory
SELECT 'D',GetDate(),* FROM deleted
    -- Insert statements for trigger here

END

GO
/****** Object:  Trigger [dbo].[Transaction_Update]    Script Date: 08/11/2009 16:54:16 ******/
CREATE TRIGGER [dbo].[ProductCode_Update]
   ON  [dbo].[tblProductCode]
   AFTER UPDATE
AS
BEGIN
    -- SET NOCOUNT ON added to prevent extra result sets from
    -- interfering with SELECT statements.
    SET NOCOUNT ON;
    INSERT INTO tblProductCodeHistory
SELECT 'U',GetDate(),* FROM deleted
    -- Insert statements for trigger here

END

 

Once you’ve done this once then just do a search and replace to create the next set of trigger e.g. replace ProductCode in the above script with MyTable.

One thing to bear in mind is if you have a username in the original table then you might want to update this as part of the delete stored proc so the last username gets in the history table.

A colleague of mine has been promising to create a script to create this off the sysobjects – but I’m not holding my breath!