Using Automation to Simplify SQL Server Management Greg RobidouxEdgewood Solutionsgregr@edgewoodsolutions.comBullet ManaleIderawww.idera.com
Idera Solutions  for SQL ServerPerformance & AvailabilityCompliance & SecurityChange Management  Backup & RecoveryAdministration
AgendaWhy AutomateWhat to AutomateHow to AutomateSQL Server ToolsOtherToolsQuestions / Wrap Up3
Why automateRedundant tasksOnly want to know when there are issuesAutomatic recoveryReduce manual stepsCreate a repeatable process4
What to automateBackupsIntegrity ChecksIndex MaintenanceLink Server StatusFree Disk SpaceTransaction Log SizeReplication AlertsDatabase Mirroring AlertsReading SQL Server Error LogScripts to Create ScriptsHigh AvailabilityGathering Performance StatisticsTracePerfmonDMVs5
What to collectBackups – failed and successfulReplication ErrorsMaintenance TasksFailed LoginsSQL Server ErrorsServer Status6
ProcessSetupSetup key components such as Database Mail, Operators, etc…ScriptsCreate scripts or sets of code to gather dataAutomateSchedule process to run and collect data. This could be something you schedule or something that occurs automatically within SQL Server.AnalyzeAnalyze data and determine what to do.NotifySend notification to either a person or another process.7
NotificationsDatabase MailSQL Agent NotificationsAlertsOperatorsOther Tools8
Database MailUses SMTPSetup Default Profilesp_send_dbmailhttp://www.mssqltips.com/tip.asp?tip=1736http://www.mssqltips.com/tip.asp?tip=1100http://www.mssqltips.com/tip.asp?tip=12619
SQL Agent Alert SystemTo send out notifications for scheduled jobs you need to enable a mail profile.10
AutomateOptionsMaintenance PlansScriptingT-SQLPowerShellWindows ScriptSQLCMDSSISEtc…Scheduling11
Maintenance PlansAllows you to automate routine tasks such as: BackupsIndex maintenanceIntegrity checksCleanup tasksCreates SSIS PackagesLimited control12
Maintenance Plan TasksBackup Database TaskCheck Database Integrity TaskExecute SQL Server Agent TaskExecute T-SQL Statement TaskHistory Cleanup TaskMaintenance Cleanup TaskNotify Operator TaskRebuild Index TaskReorganize Index TaskShrink Database TaskUpdate Statistics Task13
Maintenance PlansMaintenance Plans Usemaster.dbo.xp_create_subdirmsdb.dbo.sp_delete_backuphistorymsdb.dbo.sp_purge_jobhistorymsdb.dbo.sp_maintplan_delete_logmaster.dbo.xp_delete_filemsdb.dbo.sp_notify_operatorBACKUP…ALTER INDEX…DBCC CHECKDBDBCC SHRINKDATABASEmsdb.dbo.sp_start_jobUPDATE STATISTICS14
Maintenance Plans – Other OptionsSubplansReportingScheduling15
ScriptingMore work, but gives you more control.Stored Procedures (T-SQL or CLR)SSIS PackagesVBScriptVB.Net or C#PowerShellSMO (SQL Management Objects)SQLCMDUse of DMVs16
Scripting ExamplesIn addition to Maintenance PlansCheck Free Disk SpaceTransaction Log UsageLast Backup InfoReading Error LogsSelectively Rebuild or Reorganize IndexesScheduled Job FailuresBackup FailuresLogin FailuresScripts that Generate Scripts17
Backup All Databases ScriptDECLARE @name VARCHAR(50) -- database name  DECLARE @path VARCHAR(256) -- path for backup files  DECLARE @fileName VARCHAR(256) -- filename for backup  DECLARE @fileDate VARCHAR(20) -- used for file name SET @path = 'C:\Backup\'  SELECT @fileDate = CONVERT(VARCHAR(20),GETDATE(),112) DECLARE db_cursor CURSOR FOR  SELECT name FROM master.dbo.sysdatabases WHERE name NOT IN ('master','model','msdb','tempdb')  OPEN db_cursor   FETCH NEXT FROM db_cursor INTO @name   WHILE @@FETCH_STATUS = 0   BEGIN          SET @fileName = @path + @name + '_' + @fileDate + '.BAK'         BACKUP DATABASE @name TO DISK = @fileName         FETCH NEXT FROM db_cursor INTO @name   END   CLOSE db_cursor   DEALLOCATE db_cursorhttp://www.mssqltips.com/tip.asp?tip=107018
Log Space Usage ScriptCREATE TABLE ##logspace(databaseNamesysname,logsize decimal(10,5),logused decimal(10,5),status int)INSERT INTO ##logspaceEXEC ('dbccsqlperf(logspace)')EXEC msdb.dbo.sp_send_dbmail	@profile_name = 'SQLMail Profile',    @recipients = 'gregr@edgewoodsolutions.com',    @query = 'SELECT * FROM ##logspace WHERE logused > 75' ,    @subject = 'Log Space Usage‘DROP TABLE ##logspace19
 Free Drive Space ScriptUse sys.xp_fixeddrivesCREATE TABLE #drivespace(drive varchar(20),freespacebigint)INSERT INTO #drivespaceEXEC sys.xp_fixeddrivesSELECT * FROM #drivespaceWHERE freespace < 10000DROP TABLE #drivespace20
Scripts that Generate ScriptsUse system Meta DataStored proceduresIndex drops and creationsConstraints drops and creationsCreate insert statementsSSMS Scriptinghttp://www.mssqltips.com/tip.asp?tip=1376http://vyaskn.tripod.com/code.htm#inserts21
Reading Error Logs	Type of System LogsDatabase MailSQL AgentSQL ServerWindows22
SP_ReadErrorLogEXEC sp_readerrorlogParametersValue of error log file you want to read: 0 = current, 1 = Archive #1, 2 = Archive #2, etc... Log file type: 1 or NULL = error log, 2 = SQL Agent log Search string 1: String one you want to search for Search string 2: String two you want to search for to further refine the resultsEXEC sp_readerrorlog 0, 1, ‘Error’23
XP_ReadErrorLogEXEC xp_readerrorlogParametersValue of error log file you want to read: 0 = current, 1 = Archive #1, 2 = Archive #2, etc... Log file type: 1 or NULL = error log, 2 = SQL Agent log Search string 1: String one you want to search for Search string 2: String two you want to search for to further refine the resultsSearch from  start time Search to end timeSort order for results: N'asc' = ascending, N'desc' = descendinghttp://www.mssqltips.com/tip.asp?tip=1476http://www.mssqltips.com/tip.asp?tip=173524
ScriptingAnother way to read the error logs is to read line by line and searching for keywords.Can be done using any programming language.This tip shows how it can be done using Windows Scripting http://www.mssqltips.com/tip.asp?tip=1307Another tool is the Log Parser tool from Microsofthttp://www.microsoft.com/downloads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en25
SchedulingSQL AgentWindows Scheduled TasksVisualCronOther Third Party ToolsSQL Express – does not include SQL Agent26
SQL AgentCan RunActiveX, CmdExec, Replication, SSAS, SSIS, T-SQLRun when SQL Agent startsSQL Agent Alert Systemsp_start_jobSystem TablesNotificationsLoggingAlerts Can Start Job27
Scheduling – FrequencyBackups - DailyIntegrity Checks - WeeklyIndex Maintenance - WeeklyOther Tasks – DependsReports – Daily, Weekly28
Multi Server AdministrationMaster / TargetManage all scheduled jobs from one server29
AlertsReplicationDatabase MirroringBackupsUser DefinedEtc…ResponseExecute a JobNotify Operatorhttp://www.mssqltips.com/tip.asp?tip=939http://www.mssqltips.com/tip.asp?tip=156430
Central Management ServerNew in SQL 2008Allows you to register servers and fire off same query on all serversUses Windows Authentication OnlyCan be used for SQL 2000, 2005 and 2008http://www.mssqltips.com/tip.asp?tip=176731
Other ToolsTracePerfmonDMVshttp://www.mssqltips.com/tip.asp?tip=1035http://www.mssqltips.com/tip.asp?tip=1211http://www.mssqltips.com/tip.asp?tip=1722http://www.mssqltips.com/category.asp?catid=3132
Startup ProceduresUSE MASTERGOEXEC SP_PROCOPTION ‘SP_LOG_SERVER_START’, 'STARTUP', 'ON'GOUSE MASTERGOEXEC SP_PROCOPTION ‘SP_LOG_SERVER_START’, 'STARTUP', ‘OFF'GOhttp://www.mssqltips.com/tip.asp?tip=157433
Questions and Wrap-upThanks to our sponsor: IderaNext webcast in the series:  “Under the Hood with SQL Server Fundamentals”Don JonesJuly 8th, 2009, 4pm EDThttps://www2.gotomeeting.com/register/449103978Download SQL diagnostic manager

Sql Automation 20090610

  • 1.
    Using Automation toSimplify SQL Server Management Greg RobidouxEdgewood Solutionsgregr@edgewoodsolutions.comBullet ManaleIderawww.idera.com
  • 2.
    Idera Solutions for SQL ServerPerformance & AvailabilityCompliance & SecurityChange Management Backup & RecoveryAdministration
  • 3.
    AgendaWhy AutomateWhat toAutomateHow to AutomateSQL Server ToolsOtherToolsQuestions / Wrap Up3
  • 4.
    Why automateRedundant tasksOnlywant to know when there are issuesAutomatic recoveryReduce manual stepsCreate a repeatable process4
  • 5.
    What to automateBackupsIntegrityChecksIndex MaintenanceLink Server StatusFree Disk SpaceTransaction Log SizeReplication AlertsDatabase Mirroring AlertsReading SQL Server Error LogScripts to Create ScriptsHigh AvailabilityGathering Performance StatisticsTracePerfmonDMVs5
  • 6.
    What to collectBackups– failed and successfulReplication ErrorsMaintenance TasksFailed LoginsSQL Server ErrorsServer Status6
  • 7.
    ProcessSetupSetup key componentssuch as Database Mail, Operators, etc…ScriptsCreate scripts or sets of code to gather dataAutomateSchedule process to run and collect data. This could be something you schedule or something that occurs automatically within SQL Server.AnalyzeAnalyze data and determine what to do.NotifySend notification to either a person or another process.7
  • 8.
    NotificationsDatabase MailSQL AgentNotificationsAlertsOperatorsOther Tools8
  • 9.
    Database MailUses SMTPSetupDefault Profilesp_send_dbmailhttp://www.mssqltips.com/tip.asp?tip=1736http://www.mssqltips.com/tip.asp?tip=1100http://www.mssqltips.com/tip.asp?tip=12619
  • 10.
    SQL Agent AlertSystemTo send out notifications for scheduled jobs you need to enable a mail profile.10
  • 11.
  • 12.
    Maintenance PlansAllows youto automate routine tasks such as: BackupsIndex maintenanceIntegrity checksCleanup tasksCreates SSIS PackagesLimited control12
  • 13.
    Maintenance Plan TasksBackupDatabase TaskCheck Database Integrity TaskExecute SQL Server Agent TaskExecute T-SQL Statement TaskHistory Cleanup TaskMaintenance Cleanup TaskNotify Operator TaskRebuild Index TaskReorganize Index TaskShrink Database TaskUpdate Statistics Task13
  • 14.
    Maintenance PlansMaintenance PlansUsemaster.dbo.xp_create_subdirmsdb.dbo.sp_delete_backuphistorymsdb.dbo.sp_purge_jobhistorymsdb.dbo.sp_maintplan_delete_logmaster.dbo.xp_delete_filemsdb.dbo.sp_notify_operatorBACKUP…ALTER INDEX…DBCC CHECKDBDBCC SHRINKDATABASEmsdb.dbo.sp_start_jobUPDATE STATISTICS14
  • 15.
    Maintenance Plans –Other OptionsSubplansReportingScheduling15
  • 16.
    ScriptingMore work, butgives you more control.Stored Procedures (T-SQL or CLR)SSIS PackagesVBScriptVB.Net or C#PowerShellSMO (SQL Management Objects)SQLCMDUse of DMVs16
  • 17.
    Scripting ExamplesIn additionto Maintenance PlansCheck Free Disk SpaceTransaction Log UsageLast Backup InfoReading Error LogsSelectively Rebuild or Reorganize IndexesScheduled Job FailuresBackup FailuresLogin FailuresScripts that Generate Scripts17
  • 18.
    Backup All DatabasesScriptDECLARE @name VARCHAR(50) -- database name  DECLARE @path VARCHAR(256) -- path for backup files  DECLARE @fileName VARCHAR(256) -- filename for backup  DECLARE @fileDate VARCHAR(20) -- used for file name SET @path = 'C:\Backup\'  SELECT @fileDate = CONVERT(VARCHAR(20),GETDATE(),112) DECLARE db_cursor CURSOR FOR  SELECT name FROM master.dbo.sysdatabases WHERE name NOT IN ('master','model','msdb','tempdb')  OPEN db_cursor   FETCH NEXT FROM db_cursor INTO @name   WHILE @@FETCH_STATUS = 0   BEGIN          SET @fileName = @path + @name + '_' + @fileDate + '.BAK'         BACKUP DATABASE @name TO DISK = @fileName         FETCH NEXT FROM db_cursor INTO @name   END   CLOSE db_cursor   DEALLOCATE db_cursorhttp://www.mssqltips.com/tip.asp?tip=107018
  • 19.
    Log Space UsageScriptCREATE TABLE ##logspace(databaseNamesysname,logsize decimal(10,5),logused decimal(10,5),status int)INSERT INTO ##logspaceEXEC ('dbccsqlperf(logspace)')EXEC msdb.dbo.sp_send_dbmail @profile_name = 'SQLMail Profile', @recipients = 'gregr@edgewoodsolutions.com', @query = 'SELECT * FROM ##logspace WHERE logused > 75' , @subject = 'Log Space Usage‘DROP TABLE ##logspace19
  • 20.
    Free DriveSpace ScriptUse sys.xp_fixeddrivesCREATE TABLE #drivespace(drive varchar(20),freespacebigint)INSERT INTO #drivespaceEXEC sys.xp_fixeddrivesSELECT * FROM #drivespaceWHERE freespace < 10000DROP TABLE #drivespace20
  • 21.
    Scripts that GenerateScriptsUse system Meta DataStored proceduresIndex drops and creationsConstraints drops and creationsCreate insert statementsSSMS Scriptinghttp://www.mssqltips.com/tip.asp?tip=1376http://vyaskn.tripod.com/code.htm#inserts21
  • 22.
    Reading Error Logs Typeof System LogsDatabase MailSQL AgentSQL ServerWindows22
  • 23.
    SP_ReadErrorLogEXEC sp_readerrorlogParametersValue of errorlog file you want to read: 0 = current, 1 = Archive #1, 2 = Archive #2, etc... Log file type: 1 or NULL = error log, 2 = SQL Agent log Search string 1: String one you want to search for Search string 2: String two you want to search for to further refine the resultsEXEC sp_readerrorlog 0, 1, ‘Error’23
  • 24.
    XP_ReadErrorLogEXEC xp_readerrorlogParametersValue of errorlog file you want to read: 0 = current, 1 = Archive #1, 2 = Archive #2, etc... Log file type: 1 or NULL = error log, 2 = SQL Agent log Search string 1: String one you want to search for Search string 2: String two you want to search for to further refine the resultsSearch from  start time Search to end timeSort order for results: N'asc' = ascending, N'desc' = descendinghttp://www.mssqltips.com/tip.asp?tip=1476http://www.mssqltips.com/tip.asp?tip=173524
  • 25.
    ScriptingAnother way toread the error logs is to read line by line and searching for keywords.Can be done using any programming language.This tip shows how it can be done using Windows Scripting http://www.mssqltips.com/tip.asp?tip=1307Another tool is the Log Parser tool from Microsofthttp://www.microsoft.com/downloads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en25
  • 26.
    SchedulingSQL AgentWindows ScheduledTasksVisualCronOther Third Party ToolsSQL Express – does not include SQL Agent26
  • 27.
    SQL AgentCan RunActiveX,CmdExec, Replication, SSAS, SSIS, T-SQLRun when SQL Agent startsSQL Agent Alert Systemsp_start_jobSystem TablesNotificationsLoggingAlerts Can Start Job27
  • 28.
    Scheduling – FrequencyBackups- DailyIntegrity Checks - WeeklyIndex Maintenance - WeeklyOther Tasks – DependsReports – Daily, Weekly28
  • 29.
    Multi Server AdministrationMaster/ TargetManage all scheduled jobs from one server29
  • 30.
    AlertsReplicationDatabase MirroringBackupsUser DefinedEtc…ResponseExecutea JobNotify Operatorhttp://www.mssqltips.com/tip.asp?tip=939http://www.mssqltips.com/tip.asp?tip=156430
  • 31.
    Central Management ServerNewin SQL 2008Allows you to register servers and fire off same query on all serversUses Windows Authentication OnlyCan be used for SQL 2000, 2005 and 2008http://www.mssqltips.com/tip.asp?tip=176731
  • 32.
  • 33.
    Startup ProceduresUSE MASTERGOEXECSP_PROCOPTION ‘SP_LOG_SERVER_START’, 'STARTUP', 'ON'GOUSE MASTERGOEXEC SP_PROCOPTION ‘SP_LOG_SERVER_START’, 'STARTUP', ‘OFF'GOhttp://www.mssqltips.com/tip.asp?tip=157433
  • 34.
    Questions and Wrap-upThanksto our sponsor: IderaNext webcast in the series: “Under the Hood with SQL Server Fundamentals”Don JonesJuly 8th, 2009, 4pm EDThttps://www2.gotomeeting.com/register/449103978Download SQL diagnostic manager