With the release date for 2016 finally announced

Everyone can start gearing up to gaze upon its far shores from the 2008R2 instance they can’t or won’t upgrade for various reasons. I’m excited for a lot of the improvements and enhancements coming along, and generally hope I’m wrong about customer adoption.

One annoyance with the new release is the increase in CPU capacity for Standard Edition, with no increase in RAM capacity. You can now have up to 24 cores on your Standard Edition box. Yep, another $16k in licensing! And they’ll all be reading data from disk. Don’t kid yourself about Buffer Pool Extensions saving the day; nothing is going to beat having your data cached in memory. How many people on Standard Edition have CPU bound workloads?

Alright, now set MAXDOP and Cost Threshold to the right values. Anyone left?

Alright, check your missing index requests. Anyone left?

But Enterprise needs to be different

It’s already different. It already has a ton of features, including a plethora that smaller shops can’t or won’t ever touch. Full blown AGs, Hekaton, Page/Row Compression, ColumnStore, Online Index Create/Rebuild, Encryption, really, the list goes on and on. And c’mon, the HA/DR parts are what define Enterprise software to me.

24 cores and nothing on.

Having a fast ship is way different from having a ship that’s hard to sink.

So what’s the solution?

Microsoft needs to make money. I get it. There’s no such thing as a free etc. But do they really need to make Enterprise licensing money off of people who will never use a single Enterprise feature? Should a small shop with a lot of data really have to make a $5000 jump per core just to cache another 128-256GB of data? That seems unreasonable to me. RAM is cheap. Licensing is not.

I wouldn’t suggest à la carte pricing, because licensing is already complicated enough. What could make sense is offering higher memory limits to shops with Software Assurance. Say up to 512GB on Standard Edition. That way, Microsoft can still manage to keep the lights on, and smaller shops that don’t need all the pizzaz and razzmatazz of Enterprise Edition can still hope to cache a reasonable amount of their data.

If Microsoft doesn’t start keeping up with customer reality, customers may start seeking cheaper and less restrictive solutions.

Thanks for reading!

Brent says: Adding 8 more cores to Standard Edition answers a question no one was asking. It’s almost like raising the number of available indexes per table to 2,000 – hardly anybody’s going to actually do that, and the ones who do are usually ill-advised. (Don’t get me wrong – there’s some good stuff in 2016 Standard – but this ain’t one of ’em.)

Moving parts

SQL Server keeps on growing. With every new edition, you get more features, feature enhancements, and uh, “feature enhancements”. As I’m writing this, SQL Server 2005 is less than a week away from support ending, and SQL Server 2016 is up to RC2. Brent’s retrospective post got me thinking a bit.

We went from Log Shipping, to Log Shipping and Mirroring, to Log Shipping and Mirroring and FCIs (yeah, I know, but Clustering 2005 was a horror show), to Log Shipping and Mirroring and FCIs and AGs, and Microsoft now keeps finding ways to add Replicas and whatnot to AGs. Simple up/down monitoring on these isn’t enough.

Dumbfish

You need to make sure your servers are keeping up on about half a dozen different levels. Network, disks (even more if you’re on a SAN), CPU, memory, etc. If you’re virtualized as well, you have a whole extra layer of nonsense to involve in your troubleshooting.

And this is just for you infrastructure guys and gals.

For those of you in the perf tuning coven, you have to know exactly what happened and when. Or what’s killing you now.

Tiny bubbles

SQL Server has pretty limited memory when it comes to these things. Prior to 2016, with the advent of Query Store, and a ‘bug fix‘ to stop clearing out some index DMV usage data, your plan cache and index DMVs may not have all that much actionable or historical information on them.

And none of them keep a running log of what happened and when. Unless you have a team of highly specialized, highly paid barely cognizant familiars mashing F5 in 30 second intervals 24/7 to capture workload metrics and details, you’re not going to be able to do any really meaningful forensics on a performance hiccup or outage. Especially if some wiseguy decides the only thing that will fix it is rebooting SQL.

Monitoring is fundamental

If you have a DBA, you (hopefully) have someone who at least knows where to look during an emergency. If you don’t, it becomes even more vital to use a monitoring tool that’s looking at the right things, so you have the best set of information to work with.

There’s a learning curve on any tool, but it’s generally a lot less steep than learning how to log a Trace or Extended Events session (probably a whole mess of Extended Events sessions) to tables, and all the pertinent system DMVs, and blah blah blah. You’re already sweating and/or crying.

Because you know what’s next.

Visualizing all that data.

Time and Money

You don’t have time to do all that. You have too many servers to do all that. You need it all in once place.

SQL Sentry, Dell,  and Idera all have mature monitoring tools with lots of neat features. All of them have free trials. Just make sure you only use one at a time, and that you don’t stick the monitoring database on your production instance.

The bigger SQL gets, the more you need to keep an eye on. Monitoring just makes sense when uptime and performance are important.

Thanks for reading!

Everyone knows Implicit Conversion is bad

It can ruin SARGability, defeat index usage, and burn up your CPU like it needs some Valtrex. But what about explicit conversion? Is there any overhead? Turns out, SQL is just as happy with explicit conversion as it is with passing in the correct datatype in the first place.

Here’s a short demo:

SET NOCOUNT ON;	

SELECT
    ISNULL([x].[ID], 0) AS [ID] ,
    ISNULL(CAST([x].[TextColumn] AS VARCHAR(10)), 'A') AS [TextColumn]
INTO
    [dbo].[Conversion]
FROM
    ( SELECT TOP 1000000
        ROW_NUMBER() OVER ( ORDER BY ( SELECT NULL ) ) ,
        REPLICATE('A', ABS(CONVERT(INT, CHECKSUM(NEWID()))) % 10 + 1)
      FROM
        [sys].[messages] AS [m] ) [x] ( [ID], [TextColumn] );

ALTER TABLE [dbo].[Conversion] ADD CONSTRAINT [pk_conversion_id] PRIMARY KEY CLUSTERED ([ID]);

CREATE NONCLUSTERED INDEX [ix_text] ON [dbo].[Conversion] ([TextColumn])

One table, one million rows, two columns! Just like real life! Let’s throw some queries at it. The first one will use the wrong datatype, the second one will cast the wrong datatype as the right datatype, and the third one is our control query. It uses the right datatype.

SET STATISTICS TIME, IO ON 

DECLARE @txt NVARCHAR(10) = N'A',
@id	INT

SELECT @id = [c1].[ID]
FROM [dbo].[Conversion] AS [c1]
WHERE [c1].[TextColumn] = @txt
GO 

DECLARE @txt NVARCHAR(10) = N'A',
@id	INT

SELECT @id = [c1].[ID]
FROM [dbo].[Conversion] AS [c1]
WHERE [c1].[TextColumn] = CAST(@txt AS VARCHAR(10) )
GO 

DECLARE @txt VARCHAR(10) = 'A',
@id	INT

SELECT @id = [c1].[ID]
FROM [dbo].[Conversion] AS [c1]
WHERE [c1].[TextColumn] = @txt 
GO

The results shouldn’t surprise most of you. From statistics time and I/O, the first query is El Stinko. The second two were within 1ms of each other, and the reads were always the same over every execution. Very little CPU, far fewer reads.

Query 1:
Table 'Conversion'. Scan count 1, logical reads 738, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

SQL Server Execution Times:
CPU time = 47 ms,  elapsed time = 47 ms.

Query 2:
Table 'Conversion'. Scan count 1, logical reads 63, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

SQL Server Execution Times:
CPU time = 0 ms, elapsed time = 4 ms.

Query 3:
Table 'Conversion'. Scan count 1, logical reads 63, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

SQL Server Execution Times:
CPU time = 0 ms, elapsed time = 5 ms.

So there you go

Explicit conversion of parameter datatypes doesn’t carry any horrible overhead. Is it easier to just pass in the correct the datatype? Yeah, probably, but you might be in a position where you can’t control the parameter datatype that’s incoming, but you can CAST or CONVERT it where it touches data.

Thanks for reading!

Brent says: the key here is that we’re taking an incoming NVARCHAR variable, and casting it in our query to be VARCHAR to match the table definition. This only works if you can guarantee that the app isn’t going to pass in unicode – but in most situations, that’s true, because the same app is also responsible for inserting/updating data in this same table, so it’s already used to working with VARCHAR data. Also, just to be clear, Erik’s talking about casting the variable – NOT every row in the table. That part still blows.

You can do a lot with Extended Events

I’m really looking forward to the stuff in 2016 becoming mainstream in 2020 or so. Raise your hand if you’re using SQL Server 2014 in production. Raise your hand if your vendor supports SQL Server 2014.

Okay then.

When you open up the New Session dialog to create event session, the default screen has a nearly overwhelming amount of stuff in it. It’s good that you can search for things, but the search is kind of iffy. Not all words hit, even if you spell them right.

Where do you even begin?

Assuming things

You’d think that with Trace being deprecated and XE being the new hotness on the block, that everything would be up front to help you find what you need, expose some cool functionality, and ideally get you started trying to parse the Codd forsook hunks of XML that session data is stored in. But no. You gotta go hunting for that, too.

Ugh.

What’s in there?

Everything. Literally everything. Every weird and awesome thing you can imagine. Seriously, go look. If I started writing a list, I’d just list everything in there.

Thanks for reading!

Register now for our upcoming free SQL Server training webcasts.

This post is mostly a thought experiment

The thought was something along the lines of: I have a table I want to keep temporal history of. I don’t plan on keeping a lot of data in that table, but the history table can accumulate quite a bit of data. I want to partition the history table for easy removal of outdated data, and I want to use ColumnStore indexes because I’m just so bleeding edge all of my edges are bleeding edges from their bloody edges.

Fair warning here

This post assumes you’re already familiar with temporal tables, Partitioning, and ColumnStore indexes. I’m not going to go into detail on any of the subjects, I’m just walking through implementation. If you’re interested in temporal tables, Itzik Ben-Gan has a two part series here and here. We have a list of great Partitioning resources here, and of course, Niko Neugebauer has a (so far) 80 part series on ColumnStore over here.

On to the experiment!

The hardest part was getting the ColumnStore index on the history table. Let’s look at the process. There’s a lot of braindumping in the code. Feel free to skip the setup stuff, if you don’t care about it.

--Sample For President 2016
CREATE DATABASE [Sample2016]

--Let's not make a big deal out of this
ALTER DATABASE Sample2016 SET RECOVERY SIMPLE

--Netflix and chill
USE [Sample2016];

--Make sure this stuff is gone
IF OBJECT_ID('dbo.Rockwell') IS NOT NULL
BEGIN
ALTER TABLE [dbo].[Rockwell] SET (SYSTEM_VERSIONING = OFF);
DROP TABLE [dbo].[Rockwell]
DROP TABLE [dbo].[RockwellHistory]
END 

IF (SELECT [ps].[name] FROM [sys].[partition_schemes] AS [ps] WHERE [ps].[name] = 'TemporalDailyScheme') IS NOT NULL
BEGIN
DROP PARTITION SCHEME [TemporalDailyScheme];
END

IF (SELECT [pf].[name] FROM [sys].[partition_functions] AS [pf] WHERE [pf].[name] = 'TemporalDailyPFunc') IS NOT NULL
BEGIN
DROP PARTITION FUNCTION [TemporalDailyPFunc];
END

/*I always feel like...*/
/*Somebody's watching me...*/
CREATE TABLE [dbo].[Rockwell]
(
[ID] BIGINT,
[ProductName] NVARCHAR(50) NOT NULL,
[Price] DECIMAL(18,2) NOT NULL,
[Description] VARCHAR(8000) NOT NULL,
[AuditDateStart] DATETIME2(2) GENERATED ALWAYS AS ROW START HIDDEN NOT NULL,
[AuditDateEnd] DATETIME2(2) GENERATED ALWAYS AS ROW END HIDDEN NOT NULL,
PERIOD FOR SYSTEM_TIME ([AuditDateStart], [AuditDateEnd]),
CONSTRAINT [pk_id] PRIMARY KEY CLUSTERED ([ID]) 
) WITH (SYSTEM_VERSIONING = ON (HISTORY_TABLE = [dbo].[RockwellHistory]) )

/*Partitioning Crap*/

--Make a partitioning function
DECLARE @sql NVARCHAR(MAX) = N'', @i INT = -366, @lf NCHAR(4) = NCHAR(13) + NCHAR(10)
SET @sql = N'CREATE PARTITION FUNCTION TemporalDailyPFunc (DATETIME2(2)) AS RANGE RIGHT FOR VALUES ('
WHILE @i < 0
BEGIN
SET @sql += @lf + 'DATEADD(DAY, ' + CAST(@i AS NVARCHAR) + ', ' + CAST(CAST(GETDATE() AS DATE) AS NVARCHAR) + N' )' + N', '
SET @i += 1
END
SET @sql += @lf + 'DATEADD(DAY, ' + CAST(@i AS NVARCHAR) + ', ' + CAST(CAST(GETDATE() AS DATE) AS NVARCHAR) + N' )' + N');';
EXEC sp_executesql @sql;
--PRINT @sql
GO

--I don't have a ton of filegroups, I just want everything on Primary
CREATE PARTITION SCHEME TemporalDailyScheme
AS PARTITION TemporalDailyPFunc
ALL TO ( [PRIMARY] );

Wew

If you’re here, I should make a couple notes. Microsoft added a really cool feature to Temporal Tables recently: The ability to mark them as hidden. This is gravy for existing apps and tables, because you don’t have to store the row versioning data along with all your other data. It would be really nice if they’d add valid date ranges (read: expiration dates) to the syntax, but hey, maybe in the next RC…

I explicitly named our history table, because SQL will name it something horrible and dumb if you don’t. You don’t have much control over History table creation or indexing at conception, but you can make changes afterwards. SQL will drop a clustered index on your table that mirrors the clustered index definition of the base table.

ColumnStore Party!

So let’s see here. I have a base table. I have a history table. I have a Partitioning Scheme and Function. How does one get their history table to Partitioned and ColumnStored status? With a few catches!

First, you have to drop the index on the history table:

DROP INDEX [ix_RockwellHistory] ON dbo.RockwellHistory

The first thing I tried was just creating my Clustered ColumnStore index in place:

CREATE CLUSTERED COLUMNSTORE INDEX [cx_cs_RockwellHistory] ON [dbo].[RockwellHistory] ON TemporalDailyScheme ([AuditDateStart]);

But that throws an error!

Msg 35316, Level 16, State 1, Line 119 The statement failed because a columnstore index must be partition-aligned with the base table.

For reference, trying to create a nonclustered ColumnStore index throws the same error.

The next thing I did was create a nonclustered index, just to make sure I could create something aligned with the Partitioning. That works!

CREATE NONCLUSTERED INDEX [ix_RockwellHistory_ADS] ON [dbo].[RockwellHistory] ([AuditDateStart]) ON TemporalDailyScheme([AuditDateStart]);

Please and thank you. Everyone’s a winner. But can you create ColumnStore indexes now?

Nope. Same errors as before. Clearly, we need a clustered index here to get things aligned. The problem is, you can’t have two clustered indexes, even if one is ColumnStore and the other isn’t.

CREATE CLUSTERED INDEX [cx_RockwellHistory_ADS] ON [dbo].[RockwellHistory] ([AuditDateStart]) ON TemporalDailyScheme([AuditDateStart]);

CREATE CLUSTERED COLUMNSTORE INDEX [cx_cs_RockwellHistory] ON [dbo].[RockwellHistory] ON TemporalDailyScheme ([AuditDateStart]);

Msg 35372, Level 16, State 3, Line 121 You cannot create more than one clustered index on table ‘dbo.RockwellHistory’. Consider creating a new clustered index using ‘with (drop_existing = on)’ option.

Ooh. But that DROP_EXISTING hint! That give me an idea. Or two. Okay, two ideas. Either one works, it just depends on how uh, bottom-side retentive you are about how things are named. This will create a ColumnStore index over your clustered index, using DROP_EXISTING.

CREATE CLUSTERED COLUMNSTORE INDEX [cx_RockwellHistory_ADS] ON [dbo].[RockwellHistory] WITH (DROP_EXISTING = ON) ON TemporalDailyScheme([AuditDateStart]);

This will drop your current Clustered Index, and create your Clustered ColumnStore index in its place, just with a name that lets you know it’s ColumnStore. Hooray. Hooray for you.

DROP INDEX [cx_RockwellHistory_ADS] ON [dbo].[RockwellHistory]

CREATE CLUSTERED COLUMNSTORE INDEX [cx_cs_RockwellHistory] ON [dbo].[RockwellHistory] ON TemporalDailyScheme ([AuditDateStart]);

SUCCESS!

Never tasted so… Obtuse, I suppose. Maybe like the parts of a lobster you shouldn’t really eat. Anyway, I hope this solves a problem for someone. I had fun working out how to get it working.

I can imagine more than a few of you seeing different ways of doing this through the course of the article, either by manipulating the initial index, creating the history table separately and then assigning it to the base table, or using sp_rename to get the naming convention of choice. Sure, that’s all possible, but a lot less fun.

Thanks for reading!

Brent says: when Microsoft ships a feature, they test its operability. When you use multiple new features together, you’re testing their interoperability – the way they work together. Microsoft doesn’t always test – or document – the way every feature works together. For example, in this example, if you want to play along as a reader, your next mission is to look at query plans that span current and history data, see how the data joins together, and how it performs.

Register now for our upcoming free SQL Server training webcasts.

Starting a new job can be scary

If you’re not already established in your field, don’t know the company all that well, or taking on a role with a higher level of responsibility, it’s totally okay if you start drinking in the parking lot. Just kidding! Start drinking at home, that gives you more time to drink and bet on horses. Lifehack!

Assuming you make it into the office and don’t spend your day betting on the pony with the best name, and HR doesn’t immediately hand you substance abuse pamphlets, you begin your glorious career as Employee #2147483647. Hooray.

But will it last? Or are there things that may have you hastily editing your resume and angrily calling your recruiter by lunch?

Office Oddity

I’ve had some strange things happen to me when I started jobs (sober, I promise) that let me know exactly how long I’d be sticking around.

• Boss wasn’t sure a second monitor was in the budget
• SA password on the whiteboard by the developer cubicles
• IT contractor passed out in the hallway outside the door

First Day, Last Day

Feel free to share in the comments if you’ve had any first day deal breakers. If you’ve ever:

• Quit by lunch
• Looked at glassdoor.com after it was too late
• Found out you replaced a dead person

This is the right blog post for you!

All of these things I do
All of these things I do
To get away from you

Altered Images – “I Could Be Happy”

Brent says: I was interviewing for a DBA job, and the final interview took place in their offices. I took a tour, and one of the IT rooms was loaded with a couple dozen student desks. Each desk was barely big enough for a single small flat panel, a keyboard, and a mouse. Any team member could reach out their arms sideways and touch the person on either side of them. I didn’t even care what their jobs were, or if I’d be working in that room – I was done right there. Any company that treats any of their people that way, isn’t somewhere I wanna work.

Wanna learn from us, but can't travel? Our in-person classes now have online dates, too.

Let’s start with definitions

Many SANs have caching built in. What kind of cache is important, because if you’re dealing with non-SSD storage underneath, you could be waiting for a really long time for it to respond.

Let’s start with some definitions of the most popular caching mechanisms available for SANs. I’m not going to say ‘only’, because some vendor out there might have some proprietary stuff going on that I haven’t heard of.

Write-through: Much like synchronous Mirroring or AGs, writes have to be confirmed twice. They’ll write to the cache, but they’ll also write to the underlying disks, and then throw a secret handshake saying that it’s committed and all is well. This SUCKS if your underlying pool of disks are slow, saturated, or otherwise abused.

Write-around: Basically does this to the cache, skipping it entirely, and writing to disk. This can be fast, but then any data you write directly to disk will have to be read into cache when something needs it. If your application relies heavily on recent data, this can be a really lousy choice.

Write-back: Like good ol’ asynchronous commits, this writes to the cache, says everything is cool, and eventually writes it to hard storage. That means your most recent data is in cache and available, but maybe not on the most stable ground just yet. If you have slow disks underneath, and the power goes out before it writes to them, you could potentially lose some data here, unless your cache has some resiliency built in. So be careful what you wish for, here.

Why tempdb?

Because you people beat so much tar and sand out of it that you’re either going to strike oil or find a new dinosaur. If writes here are slow; if SQL is waiting more than 1 second for data to just write out to here, all of your subsequent reads are at the mercy of those writes.

• What’s the sense in tuning a query that will always have overhead writing to tempdb?
• Users complain that inserts are slow because you have a trigger (you know those use tempdb, right?) that stalls out for three seconds at run
• Your maintenance (DBCC CHECKDB, indexes sorted in tempdb) can’t finish because your tempdb write stalls are the envy of only a Gutenberg Press.

The moral of the story

If you’re using local storage, there’s no excuse for not going SSD.

If you went out and got yourself an expensive SAN, and now you can’t afford to put good drives in it, you’re SAN poor and you made a bad choice.

If you run tools like CrystalDiskMark or DiskSpd, you’re using SQL’s DMVs to check on disk performance, or your monitoring tool is showing bad write latency, check to see what kind of caching you’re using. Start asking questions about the underlying drives, the SAN connections, and ask for numbers from your SAN admin. Downloading more RAM won’t fix slow writes!

Thanks for reading!

Brent says: Intel’s speedy 400GB PCI Express SSDs are down in the $700 range. Just do it.

Wanna learn from us, but can't travel? Our in-person classes now have online dates, too.

Bake your own cake

Pre-cooked example databases are cool for a lot of things. The first being that everyone can have access to them, so they can follow along with your demos without building tables or inserting a bunch of data. If you mess something up, it’s easy to restore a copy. The main problem is that they were usually designed by someone who didn’t have your issues.

Most of the time it only takes a table or two to prove your point, you just need to cook up some data that doesn’t have anyone’s real information in there. With Oracle, you have a couple different options.

SQL Server-ish

Baked into every Oracle table I’ve queried thus far, are two intrinsic columns: ROWID and ROWNUM. The ROWNUM column, at least, gives us the ability to skip over generating sequential numbers with the ROW_NUMBER function. The ROWID column is a confusing string of nonsense.

AAAY LMAO

To die for

Have you ever wished you could create a table… Let’s say a demo table! And it since it’s of no consequence, SQL just wouldn’t log any changes to it? Or even that it ever existed? I mean, we have minimal logging, but what if we wanted no logging at all? Minimal logging doesn’t always work, and requires a few pre-requisites, and, well, you get the idea. If it’s high tide on St. Patrick’s Day and Jesus is eating Cobb Salad with a T-Rex in a rowboat, your inserts will be minimally logged.

Oracle can do that, with the magic of NO LOGGING!

CREATE TABLE HR.T1 
NOLOGGING --YOU CAN'T LOG ME!
AS 
SELECT ROWNUM AS "ID", 
TRUNC(MOD(ABS(ORA_HASH(SYS_GUID() )),  10000) + 1) AS "ORDER_ID",
TRUNC(MOD(ABS(ORA_HASH(SYS_GUID() )),  100) + 1) AS "SALES_REP_ID",
TRUNC(MOD(ABS(ORA_HASH(SYS_GUID() )),  1000000) + 1) AS "CUSTOMER_ID",
SYSTIMESTAMP - ROWNUM AS "ORDER_DATE",
SYSTIMESTAMP - ROWNUM + 1 AS "PROCESS_DATE",
SYSTIMESTAMP - ROWNUM + 3 AS "SHIP_DATE",
SUBSTR(SYS_GUID(), 0, 7) AS "CUST_FIRST_NAME",
SUBSTR(SYS_GUID(), 0, 10) AS "CUST_LAST_NAME",
TRUNC(MOD(ABS(ORA_HASH(SYS_GUID() )),  900000000) + 100000000) AS "CUST_PHONE",
CASE WHEN TRUNC(MOD(ABS(ORA_HASH(SYS_GUID() )), 3)) = 0 THEN 1 ELSE 0 END AS "IS_SOMETHING"
FROM ALL_OBJECTS
WHERE ROWNUM <= 10000;

I tried to make the rest of the code as close to the usual demo table SELECT INTO stuff I normally do.

• ID is an incrementing integer
• ORDER_ID is a random number between 1 and 10,000
• SALES_REP_ID is a random number between 1 and 100
• CUSTOMER_ID is a random number between 1 and 1 million
• The three date columns use the ROWNUM to subtract a span of days, and then a static number of days are added to put a little distance between each activity
• The two name columns are based on substrings of GUIDs
• CUST_PHONE is a random 9 digit number
• IS_SOMETHING is a random 1 or 0 bit column

Easy enough! And quick. 10,000 rows get inserted on my woeful VM in 0.297 ms. That’s about as long as it just took you to blink.

Of course, there are some built in Oracle goodies to generate data a little differently, but they’re (in my mind, anyway) a bit more complicated. They rely on the DMBS_RANDOM functions. There’s a lot you can do with them! The documentation is right over this way. In particular, the STRING subprogram can give you all sorts of nice junk data.

Here’s a quick example using DBMS_RANDOM.

CREATE TABLE HR.T2 
NOLOGGING --YOU CAN'T LOG ME!
AS 
SELECT ROWNUM AS "ID",
ABS(TRUNC(DBMS_RANDOM.VALUE(1,10000))) AS "ORDER_ID",
ABS(TRUNC(DBMS_RANDOM.VALUE(1,100))) AS "SALES_REP_ID",
ABS(TRUNC(DBMS_RANDOM.VALUE(1,1000000))) AS "CUSTOMER_ID",
TO_DATE(TRUNC(DBMS_RANDOM.VALUE(TO_CHAR(SYSDATE, 'J'), TO_CHAR(SYSDATE , 'J'))), 'J') AS "ORDER_DATE",
TO_DATE(TRUNC(DBMS_RANDOM.VALUE(TO_CHAR(SYSDATE, 'J'), TO_CHAR(SYSDATE, 'J'))), 'J') + 1 AS "PROCESS_DATE",
TO_DATE(TRUNC(DBMS_RANDOM.VALUE(TO_CHAR(SYSDATE, 'J'), TO_CHAR(SYSDATE, 'J'))), 'J') + 3 AS "SHIP_DATE",
DBMS_RANDOM.STRING('U', 1) || DBMS_RANDOM.STRING('L', DBMS_RANDOM.VALUE(1,6)) AS "FIRST_NAME",
DBMS_RANDOM.STRING('U', 1) || DBMS_RANDOM.STRING('L', DBMS_RANDOM.VALUE(1,10)) AS "LAST_NAME",
ABS(TRUNC(DBMS_RANDOM.VALUE(200000000,999999999))) "CUST_PHONE",
ROUND(DBMS_RANDOM.VALUE) AS "IS_SOMETHING"
FROM ALL_OBJECTS
WHERE ROWNUM <= 10000;

Quick example, he said! Alright then! The date stuff in here took me quite a while to get right. If you follow along, you have to: cast the truncated value from a range between the current system date cast as a string in Julian date format as a Julian date and… I think there’s more? I forgot this as soon as I went to bed.

But the number and string stuff is really easy! Feeding in a range of numbers is super simple. The string stuff is just one upper case character with a random-length string appended to it. These look more like names that GUID substrings, but are probably only useful to anyone trying to come up with names for an entire colony of aliens.

I apologize if your name is in here.

This insert took 1.466 seconds, plus who knows how long getting date ranges figured out. Julian! JULIAN! Why I never.

So now we have some tables

We should probably add some indexes, and figure out how to join them, huh?

Those sound like good future blog post topics.

Thanks for reading!

Wanna learn from us, but can't travel? Our in-person classes now have online dates, too.

Oh, THAT relational data

Thankfully, most major platforms (mostly) follow the ANSI Standard when it comes to joins. However, not all things are created equal. Oracle didn’t have CROSS and OUTER APPLY until 12c, and I’d reckon they’re only implemented to make porting over from MS easier. It also introduced the LATERAL join at the same time, which does round about the same thing.

Here’s some familiar joins to keep you calm.

/*Inner!*/
SELECT T1.ID, T2.ID
FROM HR.T1, HR.T2 
WHERE T2.ID = T1.ID;

SELECT T1.ID, T2.ID
FROM HR.T1 T1 
JOIN HR.T2 T2 
ON (T1.ID = T2.ID);

/*Left!*/
SELECT T1.ID, T2.ID
FROM HR.T1 T1 
LEFT JOIN HR.T2 T2 
ON (T1.ID = T2.ID);

/*Right!*/
SELECT T1.ID, T2.ID
FROM HR.T1 T1 
RIGHT JOIN HR.T2 T2 
ON (T1.ID = T2.ID);

/*Full!*/
SELECT T1.ID, T2.ID
FROM HR.T1 T1 
FULL JOIN HR.T2 T2 
ON (T1.ID = T2.ID);


/*Cross and Outer Apply*/
SELECT t1.ID, x.WHATEVER
FROM HR.T1 t1
CROSS APPLY (
SELECT t2.ID * 10000000 AS "WHATEVER"
FROM HR.T2 t2
WHERE t2.ID = t1.ID
) x;

SELECT t1.ID, x.WHATEVER
FROM HR.T1 t1
OUTER APPLY (
SELECT t2.ID * 10000000 AS "WHATEVER"
FROM HR.T2 t2
WHERE t2.ID = t1.ID
) x;

That was boring, huh? It’ll all work just as you expect it to. But we’re not done! Oracle is not without a couple neat things that it wouldn’t hurt SQL Server to implement.

Using, Naturally

I think these constructs are pretty neat. The first one is a Natural Join. This is kind of like Join Roulette, in that Oracle will choose a join condition based on two tables having a column with the same name.

SELECT *
FROM HR.EMPLOYEES
NATURAL JOIN HR.DEPARTMENTS;

The other slightly more exotic join syntax I like uses USING to shorten the join condition.

SELECT *
FROM HR.T1 JOIN HR.T2
USING(ID);

You can extend the USING syntax to join multiple columns, too, which I like because it cuts down on typing.

But what else?

Oracle also has some pretty fancy syntax for dealing with hierarchies. Even with all the options, it’s about 6 universes ahead of the recursive CTEs you have to bust out in SQL Server (if you’re not using a hierarchyid, which you’re probably not).

Here’s the Norse God table I used to show that SQL Server’s recursive CTEs are still serial in 2016:

CREATE TABLE HR.NorseGods
    (
      GodID INT NOT NULL ,
      GodName NVARCHAR2 (30) NOT NULL ,
      Title NVARCHAR2 (100) NOT NULL ,
      ManagerID INT NULL 
    );

INSERT  ALL     
INTO HR.NorseGods ( GodID, GodName, Title, ManagerID )VALUES       ( 1,  'Odin',  'War and stuff', NULL)
INTO HR.NorseGods ( GodID, GodName, Title, ManagerID )VALUES       ( 2,  'Thor',  'Thunder, etc.', 1 )
INTO HR.NorseGods ( GodID, GodName, Title, ManagerID )VALUES       ( 3,  'Hel',   'Underworld!',   2 )
INTO HR.NorseGods ( GodID, GodName, Title, ManagerID )VALUES       ( 4,  'Loki',  'Tricksy',       3 )
INTO HR.NorseGods ( GodID, GodName, Title, ManagerID )VALUES       ( 5,  'Vali',  'Payback',       3 )
INTO HR.NorseGods ( GodID, GodName, Title, ManagerID )VALUES       ( 6,  'Freyja','Making babies', 2 )
INTO HR.NorseGods ( GodID, GodName, Title, ManagerID )VALUES       ( 7,  'Hoenir','Quiet time',    6 )
INTO HR.NorseGods ( GodID, GodName, Title, ManagerID )VALUES       ( 8,  'Eir',   'Feeling good',  2 )
INTO HR.NorseGods ( GodID, GodName, Title, ManagerID )VALUES       ( 9,  'Magni', 'Weightlifting', 8 )
SELECT * FROM DUAL;

Ignoring the somewhat awkward looking inserts I was experimenting with, that’ll get you the table structure. In Oracle, CONNECT BY is used to generate hierarchies. There are many built-in components that give you information about the structure of your hierarchy as well.

SELECT 
GODID, 
TITLE, 
LPAD(' ', LEVEL * 2, ' ') || GODNAME AS "GODNAME", 
MANAGERID,
LEVEL,
SYS_CONNECT_BY_PATH(GODID, '/') AS "PATH",
SYS_CONNECT_BY_PATH(NVL(MANAGERID, 0), '/') AS "PARENT_PATH"
FROM HR.NorseGods ng
--START WITH GODID = 1
START WITH MANAGERID IS NULL
CONNECT BY PRIOR GODID = MANAGERID
ORDER SIBLINGS BY GODID;

Let’s talk about some of that!

1. The first thing you may notice is that we use LPAD to give an indented structure to the names to make the chain of command more obvious.
2. Both LEVEL and SYS_CONNECT_BY are built in components you can use to see which step in the hierarchy you’re on, and how you’ve stepped through it so far.
3. I’m also using STARTWITH to dictate which part of the hierarchy I want to begin recursion at. Since this is a small table, I want everything. I can either specify GODID = 1, or MANAGERID IS NULL. In this case, both indicate Odin.
4. Here comes CONNECT BY PRIOR, which joins GODID to MANAGERID, which is the point of the whole thing.
5. Lastly, ordering SIBLINGS by either GODID or MANAGERID NULLS FIRST gives us our desired display order.

Here are the results!

Any similarity to Marvel characters that might get us sued is absolutely ridiculous.

JOIN ME NEXT TIME

Just kidding, I wouldn’t do that to you.

Having standards is important. Especially if you drink a lot. The ANSI Standard gives us a good starting point for writing code that’s portable across multiple systems. Though it will (likely) never be flawless, joins are one area you can worry a bit less about.

Thanks for reading!

Wanna learn from us, but can't travel? Our in-person classes now have online dates, too.

I probably should have written this one first

Most of these are exactly the same as in SQL Server. There are a whole bunch of interesting analytic functions, most of which should look pretty familiar to anyone who has spent time querying SQL Server. Most, if not all, can be extended to be window functions, if you need per-group analysis of any kind.

Counting!

All of this works the same as in SQL Server.

SELECT COUNT(*), COUNT(COMMISSION_PCT), COUNT(DISTINCT COMMISSION_PCT)
FROM HR.EMPLOYEES;

Oracle does have something kind of cool if you only need an approximate count of distinct values. Don’t ask me why there isn’t a similar function to get an approximate count of all values. I wasn’t invited to that meeting. This is good for really large data sets where you just need a rough of idea of the values you’re working with.

SELECT APPROX_COUNT_DISTINCT(COMMISSION_PCT)
FROM HR.EMPLOYEES;

Sums and Averages

Fun fact: under the covers, AVG is just a SUM and a COUNT anyway.

--SUM and COUNT
SELECT JOB_ID, 
SUM(SALARY) AS "DEPT_TOTAL",
COUNT(JOB_ID) AS "POSITIONS",
(SUM(SALARY) / COUNT(JOB_ID)) * .100 AS "AVG_SALARY"
FROM HR.EMPLOYEES
GROUP BY JOB_ID
ORDER BY DEPT_TOTAL DESC;

--AVG is a little easier
SELECT JOB_ID, 
SUM(SALARY) AS "DEPT_TOTAL",
COUNT(JOB_ID) AS "POSITIONS",
AVG(SALARY) * .100 AS "AVG_SALARY"
FROM HR.EMPLOYEES
GROUP BY JOB_ID
ORDER BY DEPT_TOTAL DESC;

The Max for the Minimum

You also have your MIN and MAX functions, along with the HAVING clause, to filter aggregates.

SELECT JOB_ID, MIN(SALARY), MAX(SALARY)
FROM HR.EMPLOYEES
GROUP BY JOB_ID
HAVING MIN(SALARY) > 10000
ORDER BY JOB_ID;

Something not boring

The LISTAGG function is something I’d absolutely love to have something like in SQL Server. It takes column values and gives you a list per row, delimited by the character of your choice. It’s pretty sweet, and the syntax is a lot easier to bang out than all the XML mumbo jumbo in SQL Server.

SELECT JOB_ID, LISTAGG(EMPLOYEE_ID, ', ') WITHIN GROUP (ORDER BY EMPLOYEE_ID) AS "EMPLOYEE_IDS"
FROM HR.EMPLOYEES
GROUP BY JOB_ID
ORDER BY JOB_ID;

And the puppies is staying, yo!

For reference, to do something similar in SQL Server, you need to do this:

SELECT  [e].[JobTitle] ,
        STUFF(( SELECT  ', ' + CAST([e1].[BusinessEntityID] AS VARCHAR)
                FROM    [HumanResources].[Employee] AS [e1]
                WHERE   [e1].[JobTitle] = [e].[JobTitle]
              FOR
                XML PATH('') ), 1, 2, '') AS [EMPLOYEE_IDS]
FROM    [HumanResources].[Employee] AS [e]
GROUP BY [e].[JobTitle];

Good luck remembering that!

SQL is a portable skill

Once you have the basics nailed down, and good fundamentals, working with other platforms becomes less painful. In some cases, going back is the hardest part! My knowledge of Oracle is still very entry level, but it gets easier and easier to navigate things as I go along. I figure if I keep this up, someday I’ll be blogging from my very own space station.

Thanks for reading!

Wanna learn from us, but can't travel? Our in-person classes now have online dates, too.

We’re not so different, you and I

In any database platform, you’ll have to deal with NULLs. They’re basically inescapable, even if you own an island. So let’s compare some of the ways they’re handled between Oracle and SQL Server.

Twofer

If you take a look at the two queries below, there are a couple things going on. First is the NVL function. It’s basically the equivalent of SQL Server’s ISNULL function, where it will return the second argument if the first is, well, NULL.

The second thing you may notice is the ORDER BY. In here we can do something really cool, and specify whether to put NULLs at the beginning, or end, of our results. SQL Server will just put them first, for better or worse. If you want to put them last, you need to do some dancing with the devil. Or just use a CASE expression in your ORDER by.

SELECT EMPLOYEE_ID, COMMISSION_PCT, NVL(COMMISSION_PCT, -1) AS "NULL_TEST"
FROM HR.EMPLOYEES
ORDER BY COMMISSION_PCT NULLS FIRST;

SELECT EMPLOYEE_ID, COMMISSION_PCT, NVL(COMMISSION_PCT, -1) AS "NULL_TEST"
FROM HR.EMPLOYEES
ORDER BY COMMISSION_PCT NULLS LAST;

I love stuff like this, because it gives you easy syntactic access to presentation goodies.

There’s another function, NVL2, which I haven’t quite figured out a lot of uses for, but whatever. It takes three arguments. If the first argument is NULL, it returns the third argument. If the first argument isn’t NULL, it returns the second argument.

SELECT EMPLOYEE_ID, COMMISSION_PCT, NVL2(COMMISSION_PCT, 1, 2) AS "NULL_TEST"
FROM HR.EMPLOYEES
ORDER BY COMMISSION_PCT NULLS FIRST;

The results end up something like this below.

I just learned how to do this, too.

There’s also NULLIF! Which does what you’d expect it to do: return a NULL if the two arguments match. Otherwise, it returns the first argument. Dodge those divide by zero errors like a pro.

SELECT 
NULLIF('RUMP', 'ERIK'),
NULLIF('ERIK', 'ERIK'),
NULLIF(1, -1),
NULLIF(1, 1)
FROM DUAL;

At long last, not a Rump

Last, but certainly not least, is the lovely and talented COALESCE. It’s a dead ringer for SQL Server’s implementation, as well.

SELECT
COALESCE(NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'HELLO!')
FROM DUAL;

Is it me you’re looking for?

Intentionally left blank

NULLs happen to the best of us. Three-valued logic can be sneaky. I prefer to use canary values when possible. Those are values that could never naturally occur in data (think -999999999 or something). Again, this isn’t meant to be an exhaustive piece on NULLs and NULL handling, just a toe in the water for any SQL Server people who need to start working with Oracle.

Thanks for reading!

Wanna learn from us, but can't travel? Our in-person classes now have online dates, too.

This is not a deep dive

If you’re looking for lots of internals and explanations of what happens behind the scenes, don’t read past here. I almost made a READPAST joke. It’s that kind of day. This is just a basic overview of creating some indexes and gathering statistics. Why? Because someone just paid about $47.5k for every 0.75 cores of Oracle Enterprise licensing and they probably expect some performance out of it. This isn’t MySQL. We don’t have all day to get query results.

If you remember last time, we created a couple tables of random data, HR.T1 and HR.T2. They are currently sitting in TABLESPACE, where no one can hear you scream.

The first thing you want to do is forget about clustered indexes. Oracle has Cluster Indexes, which allow frequently joined rows from separate tables to sit on the same block of data, to reduce I/O when joining tables. Oracle has Index Organized Tables, which can be defined by a Primary Key.

But that’s more than I want to bite off!

Index Gang

Creating a Primary Key ain’t too far off from SQL Server. But there are some weird points, too. For instance, when you create a Primary Key, you can let it create an associated index, specify an index to associate with, or let Oracle pick the first index it finds to associate with the Primary Key. Full disclosure: it may not be the index you’d pick, it may be the first index that has the PK column as a leading column.

Here are some examples!

ALTER TABLE HR.T1 ADD CONSTRAINT pk_t1_id PRIMARY KEY (ID);

--I created this one a little out of order to show the USING syntax
ALTER TABLE HR.T2 ADD CONSTRAINT pk_t2_id PRIMARY KEY (ID) USING INDEX IX_T2_ID_OID_CID;

If, at some point, you realize you chose the wrong Primary Key, you can drop it without dropping the index.

ALTER TABLE HR.T2 DROP CONSTRAINT pk_t2_id KEEP INDEX;

Can’t cluster this

You can also create some pretty familiar looking index structures. There’s even an online option, if you paid through the nose. You can, of course, define your index as UNIQUE for free (for now, anyway)!

CREATE INDEX IX_T1_ID_OID_CID ON HR.T1 (ID, ORDER_ID, CUSTOMER_ID);
CREATE INDEX IX_T2_ID_OID_CID ON HR.T2 (ID, ORDER_ID, CUSTOMER_ID);

CREATE INDEX IX_T1_ID_OID_CID ON HR.T1 (ID, ORDER_ID, CUSTOMER_ID) ONLINE;
CREATE INDEX IX_T2_ID_OID_CID ON HR.T2 (ID, ORDER_ID, CUSTOMER_ID) ONLINE;

CREATE UNIQUE INDEX IX_T1_ID_OID_CID ON HR.T1 (ID, ORDER_ID, CUSTOMER_ID) ONLINE;
CREATE UNIQUE INDEX IX_T2_ID_OID_CID ON HR.T2 (ID, ORDER_ID, CUSTOMER_ID) ONLINE;

But man oh man, the best part of this to me brings in a little something from when we created the tables and test data! Creating indexes with no logging! Creating indexes online with no logging is like perf tuning God mode. Oracle for the IDDQD!

ALTER TABLE HR.T1 ADD CONSTRAINT pk_t1_id PRIMARY KEY (ID) NOLOGGING;

CREATE UNIQUE INDEX IX_T1_CID_PHN_NL ON HR.T1 (CUSTOMER_ID, CUST_PHONE) ONLINE NOLOGGING;

Other options

Oracle doesn’t exactly have filtered indexes. They have function based indexes, but to my SQL Server soaked brain, they seem more like a computed column with an index on it than a filtered index.

CREATE UNIQUE INDEX IX_T2_OD_PD_SD ON HR.T2 (UPPER(FIRST_NAME)) ONLINE;

You can also create bitmap indexes, which are good for low density columns. That’s fancy talk for ‘not very unique’. Our bit column would fall into that category. Other entrants would be stuff like gender, marital status, or Favorite Rebecca Black Song would also probably qualify.

CREATE BITMAP INDEX IX_T1_BM_ISS ON HR.T1 (IS_SOMETHING);

Ain’t no STATMAN here

To create, update, or otherwise manage statistics you use the DBMS_STATS package. It has subprograms for so many things, it’s hard to list them all. Oracle treats statistics much more importantly than SQL Server does, and with good reason: THEY ARE!

I also like the advice that the Oracle crowd has had on index fragmentation, since around 2002:

My opinion — 99.9% of all reorgs, rebuilds, etc are a total and utter waste of time and
energy. We spend way way way too much time losing sleep over this non-event.
If you are going to spend time on this exercise — make sure you come up with a way to
MEASURE what you’ve just done in some quanitative fashion you can report to your mgmt
(eg: these rebuilds I spend X hours a week doing save us from doing X IO’s every day, or
let us do Y more transactions then otherwise possible, or …..) No one, but no one,
seems to do that (keep metrics). They just feel “it must be better”. Who knows — you
may actually be DECREASING performance!! (you’ll never know until you measure)

If we wanted to gather statistics on all columns in our T1 and T2 tables, we could run commands like this:

BEGIN
DBMS_STATS.GATHER_TABLE_STATS(
'HR',
'T1',
METHOD_OPT => 'FOR ALL COLUMNS SIZE AUTO'
);
END

BEGIN
DBMS_STATS.GATHER_TABLE_STATS(
'HR',
'T2',
METHOD_OPT => 'FOR ALL COLUMNS SIZE AUTO'
);
END;

You can check on Oracle statistics in the GUI, and see that they provide pretty commensurate information to SQL Server’s statistics.

That’s a thick milkshake.

I’ll revisit this down the line

But there’s a lot I want to explore here, first. Hopefully you learned a few things along the way. I know I did writing this!

Thanks for reading!

Wanna learn from us, but can't travel? Our in-person classes now have online dates, too.

Brent’s in class this week!

So you get me instead. You can just pretend I’m Brent, or that you’re Brent, or that we’re both Brent, or even that we’re all just infinite recursive Brents within Brents. I don’t care.

Here’s the setup

A new developer has been troubleshooting a sometimes-slow stored procedure, and wants you to review their progress so far. Tell me what could go wrong here.

You are now reading this in Pat Boone’s voice.

Remember, there are no right answers! Wait…

Wanna learn from us, but can't travel? Our in-person classes now have online dates, too.

One of my least favorite things about Availability Groups

Well, really, this goes for Mirroring and Log Shipping, too. Don’t think you’re special just because you don’t have a half dozen patches and bug fixes per CU. Hah. Showed you!

Where was I? Oh yeah. I really didn’t like the backup and restore part.

You find yourself in an awkward position

When you’re dealing with large databases, you can either take an out of band COPY_ONLY backup, or wait for a weekly/daily full. But, if you’re dealing with a lot of large databases, chances are that daily fulls are out of the question. By the time a full finishes, you’re looking at a Whole Mess O’ Log Restores, or trying to work a differential into the mix. You may also find yourself having to pause backups during this time, so your restores aren’t worthless when you go to initialize things.

You sorta-kinda got some relief from this with Availability Groups, but not much. You could either take your backups as part of the Wizarding process (like Log Shipping), figure it out yourself (like Mirroring), or defer it. That is, until SQL Server 2016.

Enter Direct Seeding

This isn’t in the GUI (yet?), so don’t open it up and expect magic mushrooms and smiley-face pills to pour out at you on a rainbow. If you want to use Direct Seeding, you’ll have to script things. But it’s pretty easy! If I can do it, anyone can.

I’m not going to go through setting up a Domain Controller or Clustering or installing SQL here. I assume you’re already lonely enough to know how to do all that.

The script itself is simple, though. I’m going to create my Availability Group for my three lovingly named test databases, and add a listener. The important part to notice is SEEDING_MODE = AUTOMATIC. This will create an Availability Group called SQLAG01, with one synchronous, and one asynchronous Replica.

Critical sensitive data.

CREATE AVAILABILITY GROUP [SQLAG01]   
FOR DATABASE [Crap1], [Crap2], [Crap3]  
REPLICA ON 
N'SQLVM01\AGNODE1' WITH (ENDPOINT_URL = N'TCP://SQLVM01.darling.com:5022',  
    FAILOVER_MODE = AUTOMATIC,  
    AVAILABILITY_MODE = SYNCHRONOUS_COMMIT,   
    BACKUP_PRIORITY = 50,   
    SECONDARY_ROLE(ALLOW_CONNECTIONS = READ_ONLY),   
    SEEDING_MODE = AUTOMATIC),   
N'SQLVM02\AGNODE2' WITH (ENDPOINT_URL = N'TCP://SQLVM02.darling.com:5022',   
    FAILOVER_MODE = AUTOMATIC,   
    AVAILABILITY_MODE = SYNCHRONOUS_COMMIT,   
    BACKUP_PRIORITY = 50,   
    SECONDARY_ROLE(ALLOW_CONNECTIONS = READ_ONLY),   
    SEEDING_MODE = AUTOMATIC),   
N'SQLVM03\AGNODE3' WITH (ENDPOINT_URL = N'TCP://SQLVM03.darling.com:5022',   
    FAILOVER_MODE = MANUAL,   
    AVAILABILITY_MODE = ASYNCHRONOUS_COMMIT,   
    BACKUP_PRIORITY = 50,   
    SECONDARY_ROLE(ALLOW_CONNECTIONS = READ_ONLY),   
    SEEDING_MODE = AUTOMATIC);   
GO  

ALTER AVAILABILITY GROUP [SQLAG01]
ADD LISTENER N'SQLAGLISTEN01' (
WITH IP ((N'123.123.123.13', N'255.255.255.0')), PORT=6000);
GO

Empty inside.

The next thing we’ll have to do is join our Replicas to the AG with the GRANT CREATE ANY DATABASE permission. I prefer to do this in SQLCMD mode so I don’t have to change connections manually.

No more apple strudel!

:CONNECT SQLVM02\AGNODE2

ALTER AVAILABILITY GROUP [SQLAG01] JOIN
GO
ALTER AVAILABILITY GROUP [SQLAG01] GRANT CREATE ANY DATABASE  
GO

:CONNECT SQLVM03\AGNODE3 

ALTER AVAILABILITY GROUP [SQLAG01] JOIN
GO
ALTER AVAILABILITY GROUP [SQLAG01] GRANT CREATE ANY DATABASE  
GO

DO MY BIDDING!

Shocked, SHOCKED

And uh, that was it. I had my AG, and all the databases showed up on my two Replicas. Apart from how cool it is, it’s sort of anti-climactic that it’s so simple. People who set their first AG up using this will take for granted how simple this is.

BRB waiting for something horrible to happen.

What’s really nice here is that when you add new databases, all you have to do is add them to the Availability Group, and they’ll start seeding over to the other Replica(s). I need to do some more playing with this feature. I have questions that I’ll get into in another post in the future.

CREATE DATABASE [Crap4]
GO

ALTER AVAILABILITY GROUP SQLAG01 
ADD DATABASE [Crap4];  
GO

These are empty test databases, so everything is immediate. If you want to find out how long it will take to Direct Seed really big databases, tune in to DBA Days Part 2. If anyone makes a SQL/Sequel joke in the comments, I will publicly shame you.

Healthy green colors!

Thanks for reading!

Brent says: wanna see this capability get added to SSMS for easier replica setup? Upvote this Connect item.

Wanna learn from us, but can't travel? Our in-person classes now have online dates, too.

We love to help people plan for disasters

We’re not pessimists, we’ve just seen one too many servers go belly up in the middle of the night to think that having only one is a good idea. When people ask us to help them, we have to take a lot into consideration. The first words out of the gate are almost always “we’ve been thinking about Availability Groups”, or some bastardized acronym thereof.

Don’t get me wrong, they’re a fine thing to think about, but the problem is that usually people’s only exposure to them, if they have any exposure to them outside of thinking about them, is just in setting them up.

Usually with VMs.

On their laptop.

With a few small test databases that they just created.

And they’re really easy to set up! Heck, even I can do it.

But this is the worst way to judge how well a solution fits your team’s abilities.

Everything’s easy when it’s working

When things get hard, and when most people figure out they’re in way over their heads, is when something goes wrong. Things always wait to go wrong until you’re in production. In other words, driving a car is a lot easier than fixing a car.

If you don’t have 2-3 people who are invested mainly in the health and well-being of your Availability Groups, practicing disaster scenarios, failing over, failing back, and everything in between, you’re going to really start hating the choice you made when something goes bump in the night.

And stuff goes wrong all the time. That’s why you wanted HA/DR in the first place, right?

Stuff can go wrong when patching

I mean, REALLY wrong

Sometimes index tuning can be a pain in the neck

You need to think before you fail over

Setting them up isn’t the end of the line

Fine, don’t believe me

Play to your strengths

If you’re a team of developers, a lone accidental DBA, or simply a few infrastructure folks who don’t spend their time reading KB articles on SQL and Windows patches, testing those patches in a staging environment, and then pushing your app workload on those patches, you’re going to have a tough time.

Things that are still great:

• Log Shipping
• Mirroring
• Failover Clusters (on Windows Server 2012R2, c’mon now)

No, you don’t get all the readable replica glamour, and the databases failing over together glitz, but you also don’t get to find out you’re the first person to Google the error your Availability Group started throwing at 2am, shortly before you, your app, and all your users stopped being able to connect to it.

Try scaling up first

If part of your move to Availability Groups is Enterprise licensing, get some really fast CPUs, and enough RAM to cache all or most of your data. You may not need to offload the stuff that’s currently a headache.

Try some optimism

Optimistic isolation levels like RCSI and SI can help relieve some of the burden from large reporting queries running over your OLTP tables.

Get your script on

No, I don’t mean getting your baby mama’s name tattooed on your neck. I mean scripting out parts of failing over Mirroring or Log Shipping so that it’s not such a bleary-eyed, manual process. Availability Groups don’t keep agent jobs, users, and other custom settings synced from server to server, so you’re going to have to figure that part out anyway.

Still interested in coming up with a HA/DR solution that works for you? Drop us a line!

Thanks for reading!

Doug says: And remember folks, HA/DR solutions sometimes differ between on-premises and cloud. Make sure the cloud features you want to use are fully supported.

Wanna shape sp_Blitz and the rest of our scripts? Check out our new Github repository.

Normally I’d update the original post

But I wanted to add a bit more than was appropriate. For my interview question, I asked how you’d respond to a developer showing you progress they’d made on tuning a sometimes slow stored procedure.

While a lot of you gave technically correct answers about the recompile hint, and the filtered index, and the table variable, no one really addressed the fact that I was asking you to respond to a person that you work with about a problem on a system that you share.

To be honest, if I asked this question in an interview and someone started reading me a riot act of things that were wrong with the example, I’d be really concerned that they’re unable to work as part of a team, and that they’re not really a good fit for a lead or mentoring type role. I’m not saying you’re not technically proficient, just that I don’t want to hire the Don’t Bother Asking style DBA. I’ve been guilty of this myself at times, and I really regret it.

This is true about, and a problem for, us as a technical community. Very few people have learned everything the hard way. The nature of most SQL Server users is community and sharing oriented. Blogging, presenting, writing free scripts, etc. And that rules. If you’re interested in something, but don’t have direct experience with it, you can usually find endless information about it, or ask for help on forums like dba.se, SQL Server Central, etc. and so forth.

We’re really lucky to have way-smart people working on the same product and sharing their insights so that we don’t always have to struggle and find 10,000 ways to not make a light bulb. Or deal with XML. Whatever. Who else would have this much of an answer about making a function schemabound? Not many! Even fewer would ever find this out on their own. You would likely do what I do, and recoil in horror at the site of a scalar valued function. Pavlov was right, and he never invented a lightbulb.

Let’s look at this together

What I really wanted to get was some sense that you are able to talk to people, not just recite facts in an endless loop. When someone junior to you shows some promise, and excitement, but perhaps not the depth of knowledge you have, make some time for them. It doesn’t have to be the second an email comes through. Let’s not pretend that every second of being a DBA is a white-knuckled, F5 bashing emergency. You can spare 30 minutes to sit down and talk through that little bit of code instead of side-eyeing your monitoring dashboard.

That’s far more powerful than just telling them everything that’s wrong with what they’ve spent a chunk of their time working on.

Acknowledging effort is powerful

“Hey! You’ve really been cranking on this!” or “Cool, those are some interesting choices.” or at least leading with some positive words about their attempt to make things better is a far more appropriate way to start a conversation with a co-worker than pointing out issues like you had to parse, bind, optimize, and execute the thing yourself.

They may not be right about everything, or maybe anything, but if you just shut them down, they’ll start shutting you out. That does not make for good morale, and they won’t be the only people who notice.

Make an effort

When you spend most of your time in front of a computer, you start to forget that there are actual people on the other end. If they’re coming to you for help, guidance, or even just to show you something, it’s a sign of respect. Don’t waste it by being Typical Tech person.

Thanks for reading!

Angie says:  As the only team member to most recently be a Junior DBA, I’d like to point out how much I appreciated it when my mentors came to MY desk to watch me try and do something, or when they locked their computer when I was at their desk with questions so it was clear that I had their full attention.  It’s the little things that make the most impact sometimes!

Wanna shape sp_Blitz and the rest of our scripts? Check out our new Github repository.

This post covers two scenarios

You either created a database, and the sync failed for some reason, or a database stopped syncing. Our setup focuses on one where sync breaks immediately, because whatever it’s my blog post. In order to do that, I set up a script to create a bunch of databases, hoping that one of them would fail. Lucky me, two did! So let’s fix them.

You wimp.

You have to be especially vigilant during initial seeding

Automatic failover can’t happen while databases sync up. The AG dashboard reports an unhealthy state, so failover is manual. The good news is that in the limited test scenarios I checked out, Direct Seeding to Replicas will pick back up when the Primary is back online, but if anything really bad happens to your Primary, that may not be the warmest or fuzziest news.

Here’s our database stuck in a restoring state.

Poor Crap903

Now let’s look in the error log. Maybe we’ll have something good there. On the Primary…

Unknown,The mirror database "Crap903" has insufficient transaction log data to preserve the log backup chain of the principal database.  This may happen if a log backup from the principal database has not been taken or has not been restored on the mirror database.

Okie dokie. Good to know. On the Replica, you’ll probably see something like this…

Automatic seeding of availability database 'Crap903' in availability group 'SQLAG01' failed with an unrecoverable error. Correct the problem then issue an ALTER AVAILABILITY GROUP command to set SEEDING_MODE = AUTOMATIC on the replica to restart seeding.

Oh, correct the problem. You hear that, guys? Correct the problem.

IF ONLY I’D THOUGHT OF CORRECTING THE PROBLEM.

Sheesh

So what do we do? We can check out the AG dashboard, see a bunch of errors, and then focus in on them.

Sit, DBA, sit. Good DBA.

Alright, let’s see what we can do! We can run a couple magical DBA commands and see what happens.

ALTER DATABASE [Crap903] SET HADR RESUME

ALTER DATABASE [Crap903] SET HADR AVAILABILITY GROUP = SQLAG01;

Oh come on.

THE SALES GUY SAID THIS WOULD BE SO EASY WTF SALES GUY

The two errors were:
Msg 35242, Level 16, State 16, Line 1
Cannot complete this ALTER DATABASE SET HADR operation on database ‘Crap903’.
The database is not joined to an availability group. After the database has joined the availability group, retry the command.

And then

Msg 1412, Level 16, State 211, Line 1
The remote copy of database “Crap903” has not been rolled forward to a point in time that is encompassed in the local copy of the database log.

Interesting! What the heck does that mean? If Brent would give me his number, I’d call and ask. I don’t understand why he won’t give me his number. Well, let’s just kick this back off. We kind of expected that not to work because of the errors we saw in the log before, but it’s worth a shot to avoid taking additional steps.

ALTER AVAILABILITY GROUP [SQLAG01] REMOVE DATABASE [Crap903]
GO

ALTER AVAILABILITY GROUP [SQLAG01] ADD DATABASE [Crap903]
GO

Right? Wrong. Digging into our DMVs and Extended Events, they’re telling us that a database with that name already exists. What’s really lousy here is that this error doesn’t appear ANYWHERE ELSE. It’s not in the dashboard, it’s not in regular, documented DMVs, nor in the XE health session. It’s only in the undocumented stuff. If you’re going to use this feature, be prepared to do a lot of detective work. Be prepared to cry.

Crud

Double crud

What we have to do is go back, remove the database from the Availability Group again, then drop it from our other Replicas. We can’t just restore over what’s already there. That would break all sorts of laws of physics and whatever else makes front squats harder to do than back squats.

Since our database is in a restoring state, it’s a few steps to recover it, set it to single user so no one does anything dumber than our AG has done, and then drop it.

Drop it like it’s crap.

When we re-add the database to our Availability Group, it should start syncing properly. Lucky for us, it did!

I’m not highly available and I’m so scared.

There’s no Tinder for databases.

I’m highly available. Call me.

New features are hard

With direct seeding, you have to be extra careful about named instances and default database creation paths. If you used named instances with default database paths to Program Files, or different drive letters and folder names, this isn’t going to work. You don’t have an option to change those things. SQL expects everything to be there in the same place across all of your Replicas. I learned that the annoying way. Several times. Troubleshooting this was weird because I still can’t track down a root cause as to why anything failed in the first place. For the record, I created 50 databases, and two of them didn’t work for some reason.

Correct the problem. Just correct the problem.

Thanks for reading!

Wanna shape sp_Blitz and the rest of our scripts? Check out our new Github repository.

As of this writing, this is all undocumented

I’m super interested in this feature, so that won’t deter me too much. There have been a number of questions since Availability Groups became a thing about how to automate adding new databases. All of the solutions were kind of awkward scripts to backup, restore, join, blah blah blah. This feature aims to make that a thing of the past.

There’s also not a ton of information about how this works, the option hasn’t made it to the GUI, and there may still be some kinks to work out. Some interesting information I’ve come across has been limited to this SAP on SQL blog post, and a Connect item by the Smartest Guy At SanDisk, Jimmy May.

The SAP on SQL Server blog post says that this feature uses the same method as Azure databases to create replicas; opening a direct data link, and Jimmy’s Connect item points to it being a backup and restore behind the scenes. The Extended Events sessions point to it being a backup and restore, so let’s look at those first.

Bring out your XML!

We’re going to need two sessions, because there are two sets of collectors, and it doesn’t make sense to lump them into one XE session. If you look in the GUI, there’s a new category called dbseed, and of course, everything is in the super cool kid debug channel.

New Extended Event Smell

Quick setup scripts are below.

CREATE EVENT SESSION [DirectSeed] ON SERVER 
ADD EVENT sqlserver.hadr_ar_controller_debug(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack)),
ADD EVENT sqlserver.hadr_automatic_seeding_failure(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack)),
ADD EVENT sqlserver.hadr_automatic_seeding_start(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack)),
ADD EVENT sqlserver.hadr_automatic_seeding_state_transition(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack)),
ADD EVENT sqlserver.hadr_automatic_seeding_success(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack)),
ADD EVENT sqlserver.hadr_automatic_seeding_timeout(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack))
ADD TARGET package0.event_file(SET filename=N'C:\XE\DirectSeed.xel',max_rollover_files=(10))
GO


CREATE EVENT SESSION [PhysicalSeed] ON SERVER 
ADD EVENT sqlserver.hadr_physical_seeding_backup_state_change(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack)),
ADD EVENT sqlserver.hadr_physical_seeding_failure(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack)),
ADD EVENT sqlserver.hadr_physical_seeding_forwarder_state_change(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack)),
ADD EVENT sqlserver.hadr_physical_seeding_forwarder_target_state_change(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack)),
ADD EVENT sqlserver.hadr_physical_seeding_progress(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack)),
ADD EVENT sqlserver.hadr_physical_seeding_restore_state_change(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack)),
ADD EVENT sqlserver.hadr_physical_seeding_schedule_long_task_failure(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack)),
ADD EVENT sqlserver.hadr_physical_seeding_submit_callback(
    ACTION(sqlserver.database_id,sqlserver.sql_text,sqlserver.tsql_stack))
ADD TARGET package0.event_file(SET filename=N'C:\XE\PhysicalSeed',max_rollover_files=(10))
GO

ALTER EVENT SESSION [DirectSeed] ON SERVER STATE = START
ALTER EVENT SESSION [PhysicalSeed] ON SERVER STATE = START

Since this is so new

I haven’t quite narrowed down which are important and which yield pertinent information yet. Right now I’m grabbing everything. In a prelude to DBA days, I’m adding the StackOverflow database. With some session data flowing in, let’s figure out what we’re looking at. XML shredding fun is up next.

To get information out of the Automatic Seeding session…

IF OBJECT_ID('tempdb..#DirectSeed') IS NOT NULL
   DROP TABLE [#DirectSeed];

CREATE TABLE [#DirectSeed]
       (
         [ID] INT IDENTITY(1, 1)
                  NOT NULL ,
         [EventXML] XML ,
         CONSTRAINT [PK_DirectSeed] PRIMARY KEY CLUSTERED ( [ID] )
       );

INSERT  [#DirectSeed]
        ( [EventXML] )
SELECT  CONVERT(XML, [event_data]) AS [EventXML]
FROM    [sys].[fn_xe_file_target_read_file]('C:\XE\DirectSeed*.xel', NULL, NULL, NULL)

CREATE PRIMARY XML INDEX [DirectSeedXML] ON [#DirectSeed]([EventXML]);

CREATE XML INDEX [DirectSeedXMLPath] ON [#DirectSeed]([EventXML])
USING XML INDEX [DirectSeedXML] FOR VALUE;

SELECT
[ds].[EventXML].[value]('(/event/@name)[1]', 'VARCHAR(MAX)') AS [event_name],			
[ds].[EventXML].[value]('(/event/@timestamp)[1]', 'DATETIME2(7)') AS [event_time],
[ds].[EventXML].[value]('(/event/data[@name="debug_message"]/value)[1]', 'VARCHAR(8000)') AS [debug_message],
/*hadr_automatic_seeding_state_transition*/
[ds].[EventXML].[value]('(/event/data[@name="previous_state"]/value)[1]', 'VARCHAR(8000)') AS [previous_state],
[ds].[EventXML].[value]('(/event/data[@name="current_state"]/value)[1]', 'VARCHAR(8000)') AS [current_state],
/*hadr_automatic_seeding_start*/
[ds].[EventXML].[value]('(/event/data[@name="operation_attempt_number"]/value)[1]', 'BIGINT') as [operation_attempt_number],
[ds].[EventXML].[value]('(/event/data[@name="ag_id"]/value)[1]', 'VARCHAR(8000)') AS [ag_id],
[ds].[EventXML].[value]('(/event/data[@name="ag_db_id"]/value)[1]', 'VARCHAR(8000)') AS [ag_id],
[ds].[EventXML].[value]('(/event/data[@name="ag_remote_replica_id"]/value)[1]', 'VARCHAR(8000)') AS [ag_remote_replica_id],
/*hadr_automatic_seeding_success*/
[ds].[EventXML].[value]('(/event/data[@name="required_seeding"]/value)[1]', 'VARCHAR(8000)') AS [required_seeding],
/*hadr_automatic_seeding_timeout*/
[ds].[EventXML].[value]('(/event/data[@name="timeout_ms"]/value)[1]', 'BIGINT') as [timeout_ms],
/*hadr_automatic_seeding_failure*/
[ds].[EventXML].[value]('(/event/data[@name="failure_state"]/value)[1]', 'BIGINT') as [failure_state],
[ds].[EventXML].[value]('(/event/data[@name="failure_state_desc"]/value)[1]', 'VARCHAR(8000)') AS [failure_state_desc]
FROM [#DirectSeed] AS [ds]
ORDER BY [ds].[EventXML].[value]('(/event/@timestamp)[1]', 'DATETIME2(7)') DESC

Every time I have to work with XML I want to go to culinary school and become a tattooed cliche on Chopped. Upside? Brent might hire me to be his personal chef. Downside? I’d only be cooking for Ernie.

Here’s a sample of what we get back

I’ve moved the ‘less interesting’ columns off to the right.

Frenemy.

These are my first clues that Jimmy is right about it being a backup and restore. One of the columns says “limit concurrent backups” and, we’re also sending file lists around. Particularly interesting is in the debug column from the hadr_ar_controller_debug item. Here’s pasted text from it.

[HADR] [Secondary] operation on replicas [58BCC44A-12A6-449B-BF33-FAAF9D1A46DD]->[F5302334-B620-4FE2-83A2-399F55AA40EF], database [StackOverflow], remote endpoint [TCP://SQLVM01.darling.com:5022], source operation [55782AB4-5307-47A2-A0D9-3BB29F130F3C]: Transitioning from [LIMIT_CONCURRENT_BACKUPS] to [SEEDING].

[HADR] [Secondary] operation on replicas [58BCC44A-12A6-449B-BF33-FAAF9D1A46DD]->[F5302334-B620-4FE2-83A2-399F55AA40EF], database [StackOverflow], remote endpoint [TCP://SQLVM01.darling.com:5022], source operation [55782AB4-5307-47A2-A0D9-3BB29F130F3C]: Starting streaming restore, DB size [-461504512] bytes, [2] logical files.

[HADR] [Secondary] operation on replicas [58BCC44A-12A6-449B-BF33-FAAF9D1A46DD]->[F5302334-B620-4FE2-83A2-399F55AA40EF], database [StackOverflow], remote endpoint [TCP://SQLVM01.darling.com:5022], source operation [55782AB4-5307-47A2-A0D9-3BB29F130F3C]: 
Database file #[0]: LogicalName: [StackOverflow] FileId: [1] FileTypeId: [0]
Database file #[1]: LogicalName: [StackOverflow_log] FileId: [2] FileTypeId: [1]

[HADR] [Secondary] operation on replicas [58BCC44A-12A6-449B-BF33-FAAF9D1A46DD]->[F5302334-B620-4FE2-83A2-399F55AA40EF], database [StackOverflow], remote endpoint [TCP://SQLVM01.darling.com:5022], source operation [55782AB4-5307-47A2-A0D9-3BB29F130F3C]: RESTORE T-SQL String for VDI Client: [RESTORE DATABASE [StackOverflow] FROM VIRTUAL_DEVICE='{AA4C5800-7192-4B77-863B-426246C0CC27}' WITH NORECOVERY, CHECKSUM, REPLACE, BUFFERCOUNT=16, MAXTRANSFERSIZE=2097152, MOVE 'StackOverflow' TO 'E:\SO\StackOverflow.mdf', MOVE 'StackOverflow_log' TO 'E:\SO\StackOverflow_log.ldf']

Hey look, a restore

While I didn’t see an explicit backup command to match, we did pick up data like this:

[HADR] [Primary] operation on replicas [58BCC44A-12A6-449B-BF33-FAAF9D1A46DD]->[571F3967-FB40-4187-BF1E-36A88458C13A], database [StackOverflow], remote endpoint [TCP://SQLVM03.darling.com:5022], source operation [AFB86269-8284-4DB1-95F9-0128EB710825]: Starting streaming backup, DB size [-461504512] bytes, [2] logical files.

A streaming backup! How cute. There’s more evidence in the Physical Seeding session, so let’s look there. Prerequisite XML horrors to follow.

IF OBJECT_ID('tempdb..#PhysicalSeed') IS NOT NULL
   DROP TABLE [#PhysicalSeed];

CREATE TABLE [#PhysicalSeed]
       (
         [ID] INT IDENTITY(1, 1)
                  NOT NULL ,
         [EventXML] XML ,
         CONSTRAINT [PK_PhysicalSeed] PRIMARY KEY CLUSTERED ( [ID] )
       );

INSERT  [#PhysicalSeed]
        ( [EventXML] )
SELECT  CONVERT(XML, [event_data]) AS [EventXML]
FROM    [sys].[fn_xe_file_target_read_file]('C:\XE\PhysicalSeed*.xel', NULL, NULL, NULL)

CREATE PRIMARY XML INDEX [PhysicalSeedXML] ON [#PhysicalSeed]([EventXML]);

CREATE XML INDEX [PhysicalSeedXMLPath] ON [#PhysicalSeed]([EventXML])
USING XML INDEX [PhysicalSeedXML] FOR VALUE;

SELECT
[ds].[EventXML].[value]('(/event/@name)[1]', 'VARCHAR(MAX)') AS [event_name],			
[ds].[EventXML].[value]('(/event/@timestamp)[1]', 'DATETIME2(7)') AS [event_time],
[ds].[EventXML].[value]('(/event/data[@name="old_state"]/text)[1]', 'VARCHAR(8000)') as [old_state],
[ds].[EventXML].[value]('(/event/data[@name="new_state"]/text)[1]', 'VARCHAR(8000)') as [new_state],
[ds].[EventXML].[value]('(/event/data[@name="seeding_start_time"]/value)[1]', 'DATETIME2(7)') as [seeding_start_time],
[ds].[EventXML].[value]('(/event/data[@name="seeding_end_time"]/value)[1]', 'DATETIME2(7)') as [seeding_end_time],
[ds].[EventXML].[value]('(/event/data[@name="estimated_completion_time"]/value)[1]', 'DATETIME2(7)') as [estimated_completion_time],
[ds].[EventXML].[value]('(/event/data[@name="transferred_size_bytes"]/value)[1]', 'BIGINT') / (1024. * 1024.) as [transferred_size_mb],
[ds].[EventXML].[value]('(/event/data[@name="transfer_rate_bytes_per_second"]/value)[1]', 'BIGINT') / (1024. * 1024.) as [transfer_rate_mb_per_second],
[ds].[EventXML].[value]('(/event/data[@name="database_size_bytes"]/value)[1]', 'BIGINT') / (1024. * 1024.) as [database_size_mb],
[ds].[EventXML].[value]('(/event/data[@name="total_disk_io_wait_time_ms"]/value)[1]', 'BIGINT') as [total_disk_io_wait_time_ms],
[ds].[EventXML].[value]('(/event/data[@name="total_network_wait_time_ms"]/value)[1]', 'BIGINT') as [total_network_wait_time_ms],
[ds].[EventXML].[value]('(/event/data[@name="is_compression_enabled"]/value)[1]', 'VARCHAR(8000)') as [is_compression_enabled],
[ds].[EventXML].[value]('(/event/data[@name="failure_code"]/value)[1]', 'BIGINT') as [failure_code]
FROM [#PhysicalSeed] AS [ds]
ORDER BY [ds].[EventXML].[value]('(/event/@timestamp)[1]', 'DATETIME2(7)') DESC

And a sampling of data…

What an odd estimated completion date.

The old state and new state columns also point to backup and restore operations. I assume the completion date points to 1600 BECAUSE THIS IS ABSOLUTE WITCHCRAFT.

Ooh! Metrics!

Ignore the smaller sizes at the bottom. I’ve clearly been doing this with a few different databases. The disk IO and network metrics are pretty awesome. Now I have to backtrack a little bit…

The SAP on SQL Server blog post talks about Trace Flag 9567 being used to enable compression. It says that it only has to be enabled on the Primary Replica to work, but even with it turned on on all three of my Replicas, the compression column says false. Perhaps, like parallel redo logs, it hasn’t been implemented yet. I tried both enabling it with DBCC TRACEON, and using it as a startup parameter. Which brings us to the next set of collectors…

DMVs

These are also undocumented, and that kind of sucks. There are two that ‘match’ the XE sessions we have.

[sys].[dm_hadr_physical_seeding_stats]
[sys].[dm_hadr_automatic_seeding]

These can be joined around to other views to get back some alright information. I used these two queries. If you have anything better, feel free to let me know.

SELECT 
ag.name as ag_name,
adc.database_name,
r.replica_server_name,
start_time, 
completion_time, 
current_state, 
failure_state_desc, 
number_of_attempts, 
failure_condition_level
FROM sys.availability_groups ag
JOIN sys.availability_replicas r ON ag.group_id = r.group_id
JOIN sys.availability_databases_cluster adc on ag.group_id=adc.group_id
JOIN sys.dm_hadr_automatic_seeding AS dhas
ON dhas.ag_id = ag.group_id
LEFT JOIN sys.dm_hadr_physical_seeding_stats AS dhpss
ON adc.database_name = dhpss.local_database_name
WHERE database_name = 'StackOverflow'
ORDER BY completion_time DESC

SELECT
database_name,
transfer_rate_bytes_per_second,
transferred_size_bytes,
database_size_bytes,
start_time_utc,
end_time_utc,
estimate_time_complete_utc,
total_disk_io_wait_time_ms,
total_network_wait_time_ms,
is_compression_enabled
FROM sys.availability_groups ag
JOIN sys.availability_replicas r ON ag.group_id = r.group_id
JOIN sys.availability_databases_cluster adc on ag.group_id=adc.group_id
JOIN sys.dm_hadr_automatic_seeding AS dhas
ON dhas.ag_id = ag.group_id
LEFT JOIN sys.dm_hadr_physical_seeding_stats AS dhpss
ON adc.database_name = dhpss.local_database_name
WHERE database_name = 'StackOverflow'
ORDER BY completion_time DESC

But we get sort of different information back in a couple places. This is part of what makes me wonder how fully formed this feature baby is. The completion estimate is in this century, heck, even this YEAR. The compression column is now a 0. Just a heads up, when I DIDN’T have Trace Flag 9567 on, that column was NULL. Turning it on changed it to 0. Heh. So uh, glad that’s… there.

I smell like tequila.

Oh look, it’s the end

I know I said it before, but I love this new feature. There’s apparently still some stuff to work out, but it’s very promising so far. I’ll post updates as I get more information, but this is about the limit of what I can get without some official documentation.

Thanks for reading!

Wanna shape sp_Blitz and the rest of our scripts? Check out our new Github repository.

From the Mailbag

In another post I did on Direct Seeding, reader Bryan Aubuchon asked if it plays nicely with TDE. I’ll be honest with you, TDE is one of the last things I test interoperability with. It’s annoying that it breaks Instant File Initialization, and mucks up backup compression. But I totally get the need for it, so I do eventually get to it.

The TL;DR here

Is that if you encrypt a database that’s already taking part in a Direct Seeding relationship, everything is fine. If you already have an encrypted database that you want to add to your Availability Group, Direct Seeding has a tough time with it.

I don’t think this is an outright attempt to push people to AlwaysEncrypted, because it has a lot of limitations.

Let’s walk through this

Because I love reader sanity checks, here we go. Microsoft tells you how to add a database encrypted with TDE to an existing Availability Group here.

wordswordswordsblahblahblah

That all sounds good! So let’s follow directions. We need a database! We also need a password, and a certificate. Alright, we can do this. We’re competent adults.

/*Create databse on acceptable path to all Replicas*/
CREATE DATABASE EncryptedCrap
 ON PRIMARY 
( NAME = 'EncryptedCrap', FILENAME = 'E:\Crap\EncryptedCrap.mdf')
 LOG ON 
( NAME = 'EncryptedCrap_log', FILENAME = 'E:\Crap\EncryptedCrap_log.ldf');
 
 /*Create key*/
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'GreatP0stBrent!' 
GO 

/*Create cert*/
CREATE CERTIFICATE EncryptedCrapCert 
WITH SUBJECT = 'If you can read this I probably got fired.'

Alright, cool. We did that. Now we have to get all up in our database and scramble its bits.

/*Get into database*/
USE EncryptedCrap  
GO  

/*Create database encryption key*/
CREATE DATABASE ENCRYPTION KEY  
WITH ALGORITHM = AES_128  
ENCRYPTION BY SERVER CERTIFICATE EncryptedCrapCert 
GO 

/*Turn encryption on*/
ALTER DATABASE EncryptedCrap SET ENCRYPTION ON

SQLCMD Appreciation Header

Few things in life will make you appreciate SQLCMD mode like working with Availability Groups. You can keep your PowerShell. $.hove-it; I’m with SQLCMD.

Stick with me through the next part. You may have to do this someday.

/*Back into master*/
USE master  
GO 

/*Backup cert to fileshare*/ 
BACKUP CERTIFICATE EncryptedCrapCert   
TO FILE = '\\Sqldc01\sqlcl1-fsw\NothingImportant\EncryptedCrap.cer'  
WITH PRIVATE KEY (FILE = '\\Sqldc01\sqlcl1-fsw\NothingImportant\EncryptedCrap.pvk' ,  
ENCRYPTION BY PASSWORD = 'GreatP0stBrent!' )  
GO

:CONNECT SQLVM02\AGNODE2

USE master  
GO  

/*Set up password*/
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'GreatP0stBrent!' 
GO 

/*Restore cert from share*/  
CREATE CERTIFICATE EncryptedCrapCert  
FROM FILE = '\\Sqldc01\sqlcl1-fsw\NothingImportant\EncryptedCrap.cer'   
WITH PRIVATE KEY (FILE = '\\Sqldc01\sqlcl1-fsw\NothingImportant\EncryptedCrap.pvk',   
DECRYPTION BY PASSWORD =  'GreatP0stBrent!');
GO

:CONNECT SQLVM03\AGNODE3

USE master  
GO  

/*Set up password*/
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'GreatP0stBrent!' 
GO 

/*Restore cert from share*/  
CREATE CERTIFICATE EncryptedCrapCert  
FROM FILE = '\\Sqldc01\sqlcl1-fsw\NothingImportant\EncryptedCrap.cer'   
WITH PRIVATE KEY (FILE = '\\Sqldc01\sqlcl1-fsw\NothingImportant\EncryptedCrap.pvk',   
DECRYPTION BY PASSWORD =  'GreatP0stBrent!');
GO

:CONNECT SQLVM01\AGNODE1

USE master
GO 

ALTER AVAILABILITY GROUP SQLAG01 ADD DATABASE EncryptedCrap
GO

What did we do?

Exactly what we did. We backed up our certificate to a network share, created a private key for it, and then on two replicas we created master passwords, and created certificates using the backup of our certificate from the primary. We did this in one SSMS window. Magical. Then we added our encrypted database to the Availability Group.

If this database weren’t encrypted, everything would probably go just fine. I say probably because, you know, computers are just the worst.

But because it is encrypted, we get some errors. On our Primary Replica, we get normal startup messages, and then messages about things failing with a transient error. Not sure what a transient error is. It forgot to tie its shoelaces before running to jump on that freight car.

Log du jour

On our Replicas, we get a different set of messages. Backup failures. Database doesn’t exist. More transient errors. This time you left an open can of pork beans by the barrel fire.

I failed college algebra, again.

Over in our Extended Events session that tracks automatic seeding, we get an error code! searching for it doesn’t really turn up much. New features. Good luck with them.

Ungoogleable errors.

One bright, shiny star of error message-y goodness shows up in our Physical Seeding Extended Event session. Look at all those potentially helpful failure codes! An individual could get a lot of useful information from those.

Attempting Helpful.

If only you weren’t being laughed at by the Gods of HA/DR. Some of the physical_seeding Extended Events have values here, but none of the automatic seeding ones do.

Feature Complete.

As of now

I don’t have a work around for this. The alternatives are to decrypt, and then re-encrypt your database after you add it, or add it the old fashioned way. Maybe something will change in the future, but as of now, these don’t appear to be compatible.

I’ve opened a Connect Item about this. I’d appreciate votes of the upward variety, if you feel so inclined.

Thanks for reading!

Wanna shape sp_Blitz and the rest of our scripts? Check out our new Github repository.

TDE is one of those things!

You either need it, and quickly learn how many things it plays the devil with, or you don’t need it, and there but for the grace of God go you. Off you go, with your compressed backups, your instant file initialization, your simple restore processes. Sod off, junior.

But Microsoft maybe listened or figured out something by accident. I don’t know which one yet, but they seem excited about it! And I am too! If you read this blog post that’s probably also being monitored closely by Chris Hansen, you’ll see why.

Backup compression now works with TDE

Cool! Great! Everyone encrypt your data and compress your backups. It’s fun. I promise.

Not satisfied with a few meek and meager data points, I set out to see if increasing Max Transfer Size also increased the degree of compression. Why? This paragraph.

It is important to know that while backing up a TDE-enable database, the compression will kick in ONLY if MAXTRANSFERSIZE is specified in the BACKUP command. Moreover, the value of MAXTRANSFERSIZE must be greater than 65536 (64 KB). The minimum value of the MAXTRANSFERSIZE parameter is 65536, and if you specify MAXTRANSFERSIZE = 65536 in the BACKUP command, then compression will not kick in. It must be “greater than” 65536. In fact, 65537 will do just good. It is recommended that you determine your optimum MAXTRANSFERSIZE through testing, based on your workload and storage subsystem. The default value of MAXTRANSFERSIZE for most devices is 1 MB, however, if you rely on the default, and skip specifying MAXTRANSFERSIZE explicitly in your BACKUP command, compression will be skipped.

It left things open ended for me. Unfortunately for me, it doesn’t help. Fortunately for you, you don’t have to wonder about it.

Check out the exxxtra large screen cap below, and we’ll talk about a few points.

I AM GIGANTIC!

First, the database without a Max Transfer Size at the bottom was a full backup I took with compression, before applying TDE. It took a little longer because I actually backed it up to disk. All of the looped backups I took after TDE was enabled, and Max Transfer Size was set, were backed up to NUL. This was going to take long enough to process without backing up to Hyper-V VM disks and blah blah blah.

The second backup up, just like the blog man said, no compression happens when you specify 65536 as the Max Transfer Size.

You can see pretty well that the difference between compressed backup sizes with and without TDE is negligible.

The most interesting part to me was the plateau in how long each backup took after a certain Max Transfer Size. Right around the 1MB mark, it hits the low 380s, and never strays very far from there afterwards. I could have tried other stuff to make this go faster, but my main interest was testing compression levels.

There you have it

Max Transfer Size doesn’t impact compression levels, but it can help duration. If you want to keep playing with switches, you can throw in Buffer Count, and try striping backups across multiple files to ‘parallelize’ output.

Thanks for reading!

Psst: early heads up - we've got a killer new statistics class and new bundles.