Backup History and Information

How long a database takes to backup is something that over the years I have been asked to get.  These requests come for different reasons, sometimes it could be to find out how much it has increased over time, sometimes it could be to see if the backup job is interfering with other jobs and sometime it isn’t about duration at all, it is more about showing the backups were completed.  Over the years I have had several auditors ask for backup history.

In order to get this information, we need to pull data from two tables in the MSDB database, backupset and backupmediafamily.

Backupset

Below is how Microsoft describes that Backupset table.  This table is in the MSDB database.

“Contains a row for each backup set. A backup set contains the backup from a single, successful backup operation.”

Since most audits usually are for a date range, the start date of the backup is how we will limit the rows returned to only the ones we need.

This table also has a column named Type.  This is where you can identify the type of backup.  Usually audits, at least the ones I have been involved in, usually only care about Full and Differential backups.  Because of this we will use this column to limit the rows returned to just that two back up types.  You can adjust as needed for your environment.

According to Microsoft’s documentation, these are the backup types you might find in this table.  This can be found here.

 

Backupmediafamily

This table has many columns, however for our purposes we will only use one, physical_device_name.  This column will store the destination of the backup.

I like to include this column because if we need to do a restore, this will help us find the proper back up.  In addition, it could also help us determine if a third part tool is being used to back up the database.

In the physical_device_name column you will usually see something like this:

C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\MSSQL\Backup\100MBDB_backup_2020_02_01_061030_4137875.trn

As you can see the above path not only identifies the destination folder, it also has the file name.

Sometimes, you will see something like below:

{FCE8C7A1-C810-4F29-A612-2D39308039E3}4

Most of the time, although not all, this is because the database if being backed up by a third-party tool such as COMMVAULT.

Here is the code that pulls the data from the two tables mentioned above. If you look at the join, you will see that the two tables are joined on the media_set_id column in both tables.

SELECT s.database_name
, m.physical_device_name ‘DestinationLocation’
, CAST(DATEDIFF(SECOND, s.backup_start_date, s.backup_finish_date) AS VARCHAR(4))
+ ‘ ‘ + ‘Seconds’ AS ‘Duration’
, s.backup_start_date
, CASE WHEN ROW_NUMBER() OVER(PARTITION BY s.database_name ORDER BY s.backup_start_date ASC) = 1
THEN 0
ELSE
DATEDIFF(DAY,LAG(s.backup_start_date, 1,0)
OVER(ORDER BY DAY(s.backup_start_date)), s.backup_start_date)
END ‘DaysSinceLastBackup’
, CASE s.[type]
WHEN ‘D’ THEN ‘Full’
WHEN ‘I’ THEN ‘Differential’
WHEN ‘L’ THEN ‘Transaction Log’
END ‘BackupType’
FROM msdb.dbo.backupset s
INNER JOIN msdb.dbo.backupmediafamily m
ON s.media_set_id = m.media_set_id
WHERE s.backup_start_date >= ’04/01/2020′
AND s.backup_start_date <= ’06/30/2020′
AND s.type IN (‘D’, ‘I’)
ORDER BY s.database_name, s.backup_start_date

The above code will return a result set similar to below.  Notice the results include, Database Name, Destination, duration, start time, Days since last backup and Backup Type.

This is much of the information that I have had to gather for various reasons over the years.

Notice the column DaysSinceLastBackup.  This uses the LAG function to calculate the number of days between the current row back up start time and the previous row start time. This can be used to help identify if there are any missed backups.  We have used this as the foundation of an alert that will email us if a backup is missing.

Over the years I have found this code to be very useful.  While this focuses mostly on full and differential backups, it can easily be modified to return other backups times as well.  However, keep in mind the impact that might have on the DaysSinceLastBackup.  If you also return transaction log backups, this number might not be as useful as you might like it to be.

Unlike a DMV, the data in these tables will persist after a reboot.

Thanks for visiting my blog!!!

 

What is the Data Type Precedence order and how is it used for Implicit Converts?

There are many data types that can be used in SQL Server.  Picking the proper data type is essential, however it is also important to know when SQL Server will automatically complete a type conversion.  When SQL Server automatically converts data types, it will use the Data Type Precedence to determine what will be the target data type for the conversion.

When does SQL Server need to automatically convert data types?  There are a number of places, however you will commonly see this when joining on columns that are not the same data type or in the WHERE clause when comparing two values that are not the same data type.

This image below is a screenshot of the list that was taken from the following Microsoft document, Click Here.

Let’s take a few minutes to go over where data types are used in SQL Server and how to gather information about the data types.  As expected you can use them when creating tables and defining columns.  Each column will have a data type assigned to it.  If you look at the image below you can see the various data types on the Production.Product table in the AdventureWorks2014 sample database.

Another way to get the data types for a particular table is to run one of the following queries.  Of course you will need to change the parameter at the end to the name of the table you are looking for.  Below are two examples. One using an information_schema view, while the other is using the sys.columns table.

SELECT TABLE_CATALOG
, TABLE_SCHEMA
, TABLE_NAME
, ORDINAL_POSITION
, DATA_TYPE
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = ‘salesorderheader’

SELECT name
, Type_name(user_type_id) AS ‘DataType’
, max_length
, precision
, scale, is_nullable
FROM sys.columns
WHERE object_id = OBJECT_ID(‘production.product’)

You will also see data types used with declaring variables and stored procedure parameters.  We declaring a variable, can you not only provide a data type but can also assign the variable a value on the same line.

DECLARE @amount INT = 20

Just an FYI, you can still use the SET keyword to assign the variable a value, as in the code below.

DECLARE @amount INT

SET @amount = 20

Now that the variable is declared and assigned a value, it can now be used.  The type conversions will happen in a few places in the TSQL.  Most commonly it will be seen in the WHERE clause when the datatype in the column is different than the data type of the variable. Using the query below as an example, the variable has been incorrectly declared as a Varchar data type, while the SalesOrderID column is an INT database.  This difference is what is forcing a type conversion when doing the comparison.

DECLARE @SorderID VARCHAR(20)

SET @SorderID = ‘43659’

SELECT *
FROM [Sales].[SalesOrderHeader]
WHERE [SalesOrderID] = @SorderID

How Do I know an Auto Type Conversion took place?  Well there are a couple of different methods.  The first method is to use an Execution Plan. Before running your query, click the button in the red box below.

When looking at an Execution plan you will see a warning sign on the operator if there is an issue.  You can see this below on the operator to the far left, it is a yellow triangle.  This can be a warning for a few things, implicit convert is just one of them.

To find out if this is indeed an implicit convert warning float your mouse over it.  When you do, you will see a popup similar to below.  This clearly shows that a convert took place.

This not only tells you that the convert happened, it also identifies the consequence of the convert.  In this case, if the Cardinality Estimate is affected in a negative manner, the query optimizer may pick a less than ideal plan.  This could lead to a performance problem.

Another approach you can use to find out if an implicit convert happened is to use the sql_variant_property function.  This function accepts two arguments, an expression and a property.  The expression is what you think it might be, this is what we would like to evaluate for the datatype.  While the property has multiple options, we are going to focus solely on using the BaseType property.  We will use this to determine the data type of an expression.  Click here for find more information about the sql_variant_property.

If you look at the code below, you will see that there are two variables being declared and then in turn being passed into the first argument of the sql_variant_property function.  Notice that the two variables are different types, one TinyInt while the other is varchar.

DECLARE @one TINYINT
DECLARE @two VARCHAR(20)

SET @one = 1
SET @two = ‘2’

SELECT SQL_VARIANT_PROPERTY(@one + @two,‘basetype’) AS ‘ResultOfExpression’
, SQL_VARIANT_PROPERTY(@one,’basetype’) AS ‘DataTypeOf @one’
, SQL_VARIANT_PROPERTY(@two,’basetype’) AS ‘DataTypeOf @two’

When you run the above query you will get the results below.  Notice that the second two column return the same data type as the one specified when each of the variables were declared.  As for the first column, I am taking a TINYINT data type and adding it to a varchar datatype.  Because the data types are different, SQL Server will automatically convert the result to the variable data types that are higher up the precedent level.  In this case the resulting value will be a TINYINT.  SQL Server converted the ‘2’ to an tinyint data type from a varchar data type.

One additional tool you can use to capture implicit converts is Extended Events.  If are not familiar with Extended Events this is just a bit outside the level of this post.  However, I will cover a few basics.  Extended Events is a tool that can be used to capture information about what is happening on your server. Here is a link of a session I did at the SQL Saturday event in Dallas in 2018.  This will give you a high level overview of how to create a session.  Click Here.  Also, Grant Fritchey has a number of great blog posts on Extended Events.  They can be found here.

If you are familiar with Extended Events, the event you will want to use is the plan_affecting_convert event.

Here is the description of this event:

“Occurs when a type convert issue affects the plan. The expression value indicates the conversion that can cause inaccurate Cardinality Estimation or prevent from using a seek based plan in the query plan choice. If performance is affected, rewriting the query could help.”

Below is an example of what the data will look like.  We are able to collect the time as well as the query.  While these additional items are not in the below image, you can also collect the host, database name and login name.

Now that we have covered what implicit converts are, you might be wondering how can I prevent the auto conversions?  This is more of an matter of design than anything else.  Below are a few suggestions that will help prevent type conversions.

      • Declare data types appropriately for variables when writing TSQL code
      • User proper data types when designing parameters for stored procedures
      • Chose proper data types for table columns

In summary, there are a number of tools that we can use to identify when implicit converts.  Once you know when they are happening you can now begin to assess if it is causing a performance issue or not.  If so, you can begin to work on a solution.

Thanks for visiting my blog!!!

How Can you Provide Feedback for Azure Data Studio

Azure Data Studio has a number of great features.  When including the extensions, it has even more functionality.  Azure Data Studio allows for the creation of extensions to add or build on to the native functionality of ADS.  The question my come up in your mind, “How do I provide Feedback?”.

Feedback is very important to the Azure Data Studio team.  While I personally do not know any member of the team, if have attended sessions presented by them and they are very open to community members providing feedback.  All in an effort to continuously improvethe tool!  Just another reason why the SQL Server community is a great one to be a part of.

If we look at ADS, there is the application itself and the extensions.  The extensions come from a number of different places.  Companies like Microsoft and RedGate are just a few.  A large number of the extensions are provided by the community. Providing feedback for the extensions can be done a number of different ways.  Most will let you report issues on their GitHub site.  Microsoft has built into ADS a place to provide feedback.  This can be found in the lower right corner of Azure Data Studio.  The smile face!!

If you have the latest version of Azure Data Studio, as of March 10, 2020, the smiley face is now a different icon. See below.

If you do not see the Smiley face, right click on the Status bar and you will see this menu.  Click the “Tweet Feedback” item.

 

Simply click it Smiley Face and you will see the form below.

As you can see there are a few pretty obvious items.  Starting with the faces, smiley and frown.  These can not only be used to help express your feelings about something, it can also dictate the number of charaters you can use.  A smiley face allows for 237 characters, while a frown face allows for 259 characters.  This will allow you to Tweet your thoughts.

However, if you don’t want to use Twitter, you have a few other options. The image below shows the two options, “Submit a bug” or “Request a missing feature”.

These will take you to different places.  Starting with “Request a missing feature”.  You will be taken to the GitHub location for Azure Data Studio.  You can then complete the following form.

If this is your first time submitting an item, you will see a few links to the right of the above form.  This is something you should read.  This gives you the guidelines for submitting an issue as well the Code of Conduct.

It is important to read both of these so you know what the rules and expectations are.

When you click “Submit a Bug” a new form will open up.  It will look like the image below.

With the first drop down box you will see three options.

      • Bug Report
      • Feature Request
      • Performance Issue

While the next drop down box, Select Source has a few options as well.  These options include the following:

      • Azure Data Studio
      • An Extension
      • I dont know

Sometimes you simply may not know what the source is.  This is where the “I don’t know” option comes into play.  When you chose ‘An Extension” an additional drop down box will appear.  This new box will list out all the extensions you have installed.

Then give the issue a title and any steps needed to reproduce the issue.  It is important to provide as much details as possible so the development team will have enough information to address the issue at hand.

Now that you have entered all the important information you are ready to click: Preview in GitHub.  When you do this, ADS gives you a pretty good description of what the issue is and how we need to proceed.

 

When you click OK, you will be asked to login to GitHub.  It is here you will need to paste what is on your clipboard or enter new comments.

Now you are ready to submit.  Hopefully your suggestion will now be used to improve the Azure Data Studio end user experience.

Thanks for visiting my blog!!

 

 

ADS: TSQL Checker

Azure Data Studio has quite a few extensions that are very useful.  Over the past six to eight months I have completed a number of posts on quite a few of these extensions.  So far one of my favorite extension is the PoorSQL Formatter.  It provides great assistance in the formatting of Transaction SQL code. What it doesn’t do, is help identify code that is not in line with best practices.  This is where the TSQL Checker comes into play.

This extension was developed by Daniel Janick. Daniel is a Microsoft MVP from the Austin, TX area.  He can be found at @SQLSME on Twitter.

You might be wondering exactly what does this extension check.  According to the extension documentation this is what it checks:

“TSQL checks for hints, select * from options”

Of course “SELECT *” is a very well known problem in query design.  Hints and options can also create significant performance issues as well.   What exactly is a hint?  According to Microsoft’s documentation a hint is something that will over ride the execution plan that the query optimizer might select for a query. Here is a link to Microsoft’s documentation on hints.  Hints will act on the four DML statements, SELECT, INSERT, UPDATE and DELETE.  Hints should only be used as a last resort and only by experienced developers or DBAs.

While this extension is still in preview mode, it can still be very useful.  To install it you will click the “Install” button, green button below.  This will appear when you are looking at the documentation of the extension.  When you do, you will be asked to open the GitHub sight for this extension.

When you click open, you will be taken to the GitHub site for this extension.  You will want to download the tsqlchecker-0.0.1.vsix file.  It is highlighted below.  When saving it, save it in a location where it will not be overwritten.

To install this extension, simply go to the File menu and then go to “Install extension from Vsix file”.  While some extensions require a restart of Azure Data Studio, this extension does not.

The purpose of this extension is to help you identify code that is not inline with best practices.  In order to do so, the extension needs to mark the offending code in some manner.  The extension will do this one of three ways.

    1.  Highlighting the code with a box:
    2.  Highlighting the code in yellow:
    3. Highlighting the code in red:

Let’s take a look at some of the code that the TSQL Checker will flag.

Note: This code was take from the Microsoft documentation.  This really isn’t a post on what these things do, just how TSQL Checker flags them.  Refer to the Microsoft documentation for more information on these.

SELECT *

MAXDOP

FORCESEEK

FORCESCAN

LOOP JOIN

MERGE JOIN

OPTIMIZE FOR

HASH JOIN

For all of these warnings, the developer has also added a popup that will provide additional information.  When you float the cursor over the warning you will see somethings similar as you is in the image below.

I really like this extension.  It reminds us of code that is less than ideal and should be reviewed.  Remember, few if any of these hints are recommended for production use.  If you run into a situation that you think these are part of the solution, just do your homework first.  Research and see if there is a better way to address the problem.

Thanks for visiting my blog!!!

 

ADS: DB Snapshot Creator

Like many Azure Data Studio extensions, DB Snapshot Creator is designed to bring functionality into ADS that is not present by default.  This extension was developed by Sean Price. As the name suggests, this extension can be used to easily create database snapshots.  Before going too deep into this extension, let’s take a quick moment to go over what a snapshot is.

First of all don’t think of it as a backup.  Think of it as a read-only copy of the database.  As pages are modified for the first time in the source database, SQL Server will move the original unmodified pages to the snapshot.  This process will allow users to see that snapshot that is a mirror of the database when the snapshot was created. You might be asking, why should I use a snapshot?  There are a number of reasons, but the biggest in my mind is the ability to use the snapshot for reporting purposes.

What if an 8-K page is modified twice?  SQL Server will only copy the page to the snapshot the first time it is modified.  After that, the page will not be copied again.

In Microsoft’s documentation, found here, right at the beginning of the document MS makes one very important point.

“Database snapshots are unrelated to snapshot backups, snapshot isolation of transactions, or snapshot replication.”

To find if there are any snapshots you can use the code below.  This code will not only tell you if snapshots exist, it will also return what database it is associated with and when it was created.

SELECT    a.name AS ‘Snapshot Name’
                , a.create_date AS ‘Snapshot Create Date’
                , b.name AS ‘Source Database’
                , b.create_date AS ‘Database Create Date’
FROM sys.databases a
     INNER JOIN sys.databases b
ON a.source_database_id = b.database_id

This is what the results will look like.

Microsoft has some good documentation on Snapshots.  Go here to read it.

How to Create the Snapshot in SSMS

This is pretty easy to do.  Many times if you want to do something with a database you can right click on the database and easily perform the desired task. This could be completing a backup, shrink the database and restore the database.  One task that is not present is Create Snapshot.  Given this, you will need to use TSQL to create the snapshot.  Note the file extension, it is not MDF or NDF.

CREATE DATABASE [AdventureWorks2014_Snapshot_02_27_2020_01] ON
(NAME = [AdventureWorks2014_Data],
FILENAME = ‘C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\MSSQL\DATA\AdventureWorks2014_Snapshot_02_27_2020_01.ss’)
AS SNAPSHOT OF [AdventureWorks2014];
Below is a screenshot of the files in the file system.  As you can see, the file for the snapshot has a different extension and there is not a corresponding .ldf file.

How to Create the Snapshot in Azure Data Studio

The option to use TSQL is always present, even in Azure Data Studio.  And just as in SSMS, when you right click on the database, Create Snapshot is not an option.  This is where the DB Snapshot Creator extension can help.  Something to keep in mind as we go over this extension, it is still in preview mode.  Neither the documentation in Azure Data Studio nor the GitHub site have a great deal of documentation on this.  However, there is an FAQ that answers three questions.  These questions are below and were taken directly from the extension documentation.

To install the extension you will first need to download the .vsix file.  When viewing the extension the in the Extension Marketplace, you can click the green Install button. This will take you to the Github where you can download the file.

Make sure the place this file in a location where it will not be deleted.  Once downloaded, you can go to the File menu and to “Install Extension from VSIX package”.

Then simply find the file and it will install.  Now when you right click on a database you will see a new menu item to create the snapshot.

By default, when you click the above menu item Azure Data Studio will write the code to create the extension, but not actually create it.  The CREATE DATABASE statement earlier in this post was created by this extension.  This code is also below.

CREATE DATABASE [AdventureWorks2014_Snapshot_02_27_2020_01] ON
(NAME = [AdventureWorks2014_Data],
FILENAME ‘C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\MSSQL\DATA\AdventureWorks2014_Snapshot_02_27_2020_01.ss’)
AS SNAPSHOT OF [AdventureWorks2014];

However, if you would like to skip the step of having the code written, you can go to preferences and uncheck the box below.

While in SSMS, the snapshots are located in a folder as seen below.

In Azure Data Studio, the snapshots are located at the same level as the rest of the databases.

 

This extension makes the creation of a database snapshot very easy!  And like all other extensions, it has a great price. FREE!!

Thanks for visiting my blog!!

 

ADS: New Database

Regardless of what tool we are using, SQL Server Management Studio or Azure Data Studio, the need to create new databases is always present. Using Transact SQL is an option in both tools.  What is not an option in both tools is to right click and to go “New Database”.  This has been in SSMS for many years, however it is NOT present in Azure Data Studio.

Below you can see what the menu options are when you right click on databases in SSMS.  The New Database option is right at the top.

 

Now lets move on to Azure Data Studio.  When you right click on a server you will have a number of options, including new query, refresh and disconnect.  What you do not see is “Delete Database”.

As with a great deal of functionality in Azure Data Studio, the ability to right click on a server and create a new database comes with the installation of an extension.  In this case the extension was created by Keven Cunnane, It is pretty simple to install and cost the same as ADS, nothing.  This extension is currently not in “Preview” mode.  It is currently at version 1.0.0.

To install you will need to click the install button when looking at the extension documentation in Azure Data Studio.

Once you do, you will be taken to the location to download the .VSIX file.  You will need to download it and save to a location where it won’t be deleted or overwritten. To install, simply click on “Install Extension from .VSIX package” on the File menu.

Once you do you will see in the lower right corner that the extension has been installed.  A restart of Azure Data Studio is not required.  Now when you right click server, what do you see?  Well..you will see that nothing changed.  You will need to right click on Databases under the connection.

Once you click “New Database” you will not see the same dialog box you will see in SSMS.  What you will see is something like what is in the image below.  In the box you will enter the name of the new database.

Then hit enter and assuming you have the proper rights, ADS will create the database.  You will not be able to enter any of the settings we often change to accommodate the specific needs of the database or system.  This includes things like file location, recovery model, file growth rate and many more. With this, you will either need to use TSQL to adjust some of the settings or use SSMS.  This method of creating a database will depend on the defaults for all the settings.

Overall this is a good extension although it depends heavily on default database settings.  I still prefer to use TSQL when creating a database, however this is a nice simple method to use as well.

Here is a list of a number of available extensions, click here.  This list does not include all the extensions that are available but is still a nice list.

Thanks for visiting my blog!!