TSMBLOG.org

If the TSM V6 active log and archive log directories are not sized adequately for the transaction load, the TSM V6 server might crash with a message that indicates the server is out of log space. This is described in APAR 1389352: Tivoli Storage Manager V6.1 server might crash if log is not sized appropriately.

This is an in-depth document that provides partial guidance in how to size your TSM V6 Active log and Archive log based on your existing TSM V5 server. Some people might be tempted to think that just throwing enough disk space at your TSM V6 server will solve the problem. But if you want to gain some in-depth knowledge and want to size your logs based on a more educated guess, take a look at the spreadsheet I've created.

This is a short blog entry, as all the information is contained in the spreadsheet itself. If you take some time and walk through it from top to bottom you will touch all bases needed to make a decent Active log and Archive log size estimate.

The spreadsheet is unprotected and fully adjustable for those who want to make any adjustments. This is v1.0.

ActivelogandArchivelogsizingspreadsheet_upgradefromTSMV5toV6.xls

With the release of TSM V6, the new reporting functionality (TSM Monitoring and Reporting) was introduced which, unlike the very much adopted and appreciated TSM Operational Reporting (TOR)) is capable of trend analysis and forecasting - because it utilizes its own separate database. However, this new piece of software comes with its own requirements, on of them being a separate server to run.

* But what if you don't need or want to install the new monitoring tool?
* If you don't need or want the new functionality for now?
* If you just to continue to use the 'good-old' TSM Operational Reporting (TOR) instead of building new reports in the new tool?
* If you just want to continue monitoring of your TSM environment on a daily basis, instead of insights in your historical data?

I've seen the requests for TOR on TSM V6.1 on the mail list, discussion boards, etc. on the internet.

Roger Stakkestad from IBM in Norway has created the document "Customization of Operational Reporting for use with TSM 6.1", which is available for download here.

UPDATE 02/12/2009 - The link above points to an early version of this document. Roger has been kind enough to mail the most recent version (revision 3.0) with reference to a 6.1 bug that sometimes creates problems for TOR:
Customization_of_Operational_Reporting_for_use_with_TSM_6_rev_3.0.doc

This document explains step-by-step how to customize the default reports and monitors in TSM 5.5.3 Operational Reporting to make them work with a TSM 6.1 server.

Thanks Roger, your work might come in handy

As the TSM V6 database is now on DB2, someday a need will arise to administrator the database from the DB2 command line or the DB2 Control Center GUI. IBM states that as a TSM administrator there is no need to become a trained DB2 DBA person. However, some APAR's already are referring to setting a DB2 configuration parameter. For example; set NUM_DB_BACKUPS to a different value. Because I couldn't easily find a proper step-by-step guidance (except for the in-depth DB2 documentation center) on how to set these specific type of values and as I'm sure (although you're not supposed to become a DB2 DBA as stated) that in the future more and more changes are needed to the configuration of the DB2 database to resolve TSM problems or tune a TSM environment, I thought that this information should be available somewhere to be helpful as a quick reference.

In the following text, I describe (step-by-step) how to connect, query and change a DB2 configuration parameter on Linux from the DB2 command line. Replace things when necessary when performing this steps on a different platform.

- Log in to the Linux system and start the DB2 command line /opt/tivoli/tsm/db2/bin/db2.
- This will open the DB2 command line utility and the prompt will change to 'db2 =>'.
- Before performing any actions it is necessary to start the 'database manager':

db2 => start database manager
DB20000I The START DATABASE MANAGER command completed successfully.

- Now you'll need to connect to a DB2 database. The default alias AND database name for a TSM database is TSMDB1. To verify this, type the command below to list all databases in the 'database directory':



In this case (and most likely the majority of a fresh TSM installation with its own, dedicated DB2 environment), there is only 1 database by the default name of TSMDB1. The 'Database alias' and 'Database name' correspond with the 'Database Name' output of a 'q db f=d':



- Now, that you know the DB2 database name, it is time to connect to it:



- Once connected, you're able to display the database configuration. Below is a small sample extracted from the output generated:

db2 => get db cfg



Scroll the output to see some interesting parameters, like the location of the different logs, etc. One of the options it the one we're after; "Number of database backups to retain (NUM_DB_BACKUPS)", which is now set to a value of 1.

- To set the NUM_DB_BACKUPS to 2:

db2 => update db cfg for TSMDB1 using num_db_backups 2
DB20000I The UPDATE DATABASE CONFIGURATION command completed successfully.

- Quit the DB2 command line to return to the OS prompt:

db2 => quit
DB20000I The QUIT command completed successfully.

'db2cc' is the GUI variant of 'db2'. After exporting your DISPLAY variable, start the DB2 Control Center using /opt/tivoli/tsm/db2/bin/db2cc) and select "Tools" - "Command Editor" from the menu, to open the GUI command line.



Connect through use of the command line statement ('connect to TSMDB1') or select TSMDB1 from the "Target" pull-down menu.
Now, you can enter the commands above.

Instead of starting up and entering into the db2 command line, you can issue a complete command from the AIX command line,
this is like going 'into' dsmc or running an entire command at once. For example; 'db2 update db cfg for TSMDB1 using num_db_backups 2'.

With the TSM Client command 'set access' you are able to give other nodes access to the backups and/or archives of the local node. You can specify ALL backups/archives, specific directories or (a) specific file(s). By granting access to another node, that node is able to restore/retrieve the data of the local node. Besides regular backup/archive data, you can also grant access for images.

For example:



After issuing the command on the node HANNIBAL_TSMCL, the node FACE_TSMCL is able to restore all backup data of HANNIBAL_TSMCL. During the 'set access' command it is checked (if you specified a specific directory or specific files) if they do exist/are previously backed up and present in the BACKUPS or ARCHIVES tables. If they don't, you can't set access (TSM client error: ANS1084E No files have previously been backed up for XX). When a wildcard (*) is used, it is assumed that data exists and no such test is performed. You'll need to set separate rules for backup or archive type data.

With the command 'query access' you can check on your setup:



With the command 'delete access' you can delete access rules. Choose the index number from the list displayed. This index number is only for human reference. For example, if you have 2 rules and you delete rule 1, then rule 2 moves to the top and becomes rule 1.



The GUI equivalent of these commands can be found in the TSM B/A GUI 'Utilities' menu as 'Node Access List'. See the screenshot below. Here you can add, change or delete the access rules.



There are several ways in TSM to let one node access the data of another node; think about VIRTUALNODENAME, Proxy authorization, change dsm.opt NODENAME and reset password, etc. But the difference between all these methods and the 'set access' method is that the latter is NOT easily visible in the TSM Server and can be issued by anyone with access to the TSM B/A Client, in most environments the system administrator of your TSM client. They can set access rules between their nodes and use TSM, for example, as a data movement tool without you knowing about it. This might be a security exposure. But luckily, there are a few ways to track down this behavior on your TSM server:

Activity Log

As the 'set/delete access' command is issued on a TSM client it is logged in the TSM servers' activity log as ANR4483/ANR4484. The 'query access' command is not logged.



Generally, most TSM shops set the ACTLOGRET to a certain period, let's say 30 days. Whether or not you have software/tooling in use to keep around all your actlog records for years, someday you'll want an easy way to see all of your 'set access' rules in one-pass. Just because you want to know about it or because of an audit requires you to produce such a list. Note that I've said that this information is not *easily* available. There is not a standard query or table you can display. But, there is a way.

Hidden table

When a node want to restore/retrieve data from another node, beginning the restore/retrieve operation the BACKUPS and ARCHIVES tables are compared with the access rules in the hidden table Authorization.Rules. The BACKUPS or ARCHIVES tables are not changed during a 'set access' command; that’s the reason that it is processed immediately. Hence, all access rules for all clients on a TSM server are saved in the hidden table Authorization.Rules, which cannot be queried directly, as there is no reference for it in TSM by default. Follow the steps below to display the content of this table. If this hidden table is empty (no data is returned and Step 1 below will fail telling you the database table is empty) than you can safely assume that there are no local ‘set access’ statements defined on any of the TSM Servers’ nodes.

1 - Create a temporary sqltable, as an overlay

tsm: BLACK>create sqltable Authorization.Rules TH
SQL table TH created to map base table Authorization.Rules.

Note that the existing table Authorization.Rules is mapped to the temporary table TH. Replace TH with the name you prefer. The table name is case sensitive. List all of your mapped tables with the command 'show sqltables'.

2 - Display the contents of the temporary table



There you have it, the contents of the hidden table Authorization.Rules. Read on for a detailed explanation of all the fields, and for a query to replace the default field names C0-C11 with more descriptive names.

3 - Drop the temporary table (let's keep things organized!)

tsm: BLACK>drop sqltable TH
SQL table TH dropped.

Hidden table 'Authorization.Rules' explained:

After setting the access rule 'set access backup * face_tsmcl' the contents of the table are:



For each seperate access rule per node, a seperate entry is listed in the table.

Explanation of the columns:



See this TSMBLOG.org Weblog article to see how to map the NODE_NAME to NODE_ID with ease.

To see the most relevant information in a human understandable format, you can use this query:

select C0 as SOURCE_NODE_ID, C1 as OWNER, C2 as COUNTER, C3 as TYPE, C4 as FSID, C5 as HL_NAME, C6 as LL_NAME, C7 as TARGET_NODE_ID from temporary-tablename

For example:



C4, C5 and C6 combined together make up the path (FILESPACE + HL_NAME + LL_NAME):

Example 1

set access backup * FACE_TSMCL



In the table, the ↑ symbol represents the wildcard *.

Example 2

set access backup c:\ FACE_TSMCL



Example 3

set access backup C:\temp\sch17.ldf FACE_TSMCL



The same logic applies to AIX filesystems.

A general remark (not very important, but just to be complete); for each access rule set on a Windows TSM client where the wildcard (*) is used 2 entries are added to the hidden table Authorization.Rules, of which the second entry has a value of 0 in the C9 field. When the same access rule is set on an AIX TSM Client only one entry is added to the table with the C9 field set to the fixed value of 28672.

One of our SQL administrators reported that the start-up of the DP for MSSQL GUI takes 2.5 minutes. It is now reduced to 5 seconds.

The reason for this spectacular improvement is listed among the "IBM Tivoli Storage Manager for Databases: Data Protection for Microsoft SQL Server Version 5.5.1 Known Problems and Limitations".

When there is no connection to the internet during start-up of the GUI it will take, in our situation, 2.5 minutes to start-up. On the SQL servers (or, the Windows servers in general) there is no connection to the internet and as a result the start-up of the GUI will be very poor. According to the link above the DP for MSSQL application is signed with and includes managed code based on the .NET framework. During start-up it tries to verify the authenticode signature online; by visiting the Certificate Revocation List (CRL) at http://crl.microsoft.com. If an internet connection is blocked or unavailable (not uncommon for a server), the outgoing HTTP requests are just dropped after a while and the GUI will start. No message is displayed to the end user, so one is just wondering why the start-up is so slow.

But the solution is simple (if not blocked by some AD policy); you can disable the online check for the CRL in Internet Explorer yourself.

- Select 'Tools' - 'Internet Options'
- Tab 'Advanced'
- Scroll down to 'Security'
- Uncheck/deselect 'Check for publisher's certificate revocation'



After this setting was saved the start-up of the DP for MSSQL GUI was reduced to just 5 seconds. This problem is not just with the DP for MSSQL GUI. It is with other software (including non-IBM), built on the .NET framework and using its signatures, as well.

The related TSMBLOG.org KB entry can be found here.

Imagine you've got multiple TSM servers (who hasn't?) which are able to communicate with each other using server-to-server communication. You now want to move a client between two TSM servers and all you want to export is the node definition (when there is no data yet or when you are able to rebuild the data on the destination TSM server) this is easy enough:

----> export node NODENAME filedata=none toserver=SERVERNAME

But what if all the data must be retained and you have lots of it, let us say 1 TB. For the sake of simplicity, I will assume this data is, compressed, on just one tape 3592-JB tape cartridge. The command, in its simplest form, would be:

----> export node NODENAME filedata=all toserver=SERVERNAME

This means that on the exporting server the volume with data belonging to this node will be mounted and on the importing server another volume will be mounted. Both the metadata and the actual backup data will be send through TCP/IP server-to-server communication. Under not so ideal circumstances loads and loads of volumes will be mounted, positioned, read, etc, etc. It is not uncommon to have to wait for hours, even days for an export session. In the post 'A plethora of exporting methods', I've described several different ways to export in such a way that enables a 'large' node to backup again within 24 hours.

Dealing with large nodes is a major issue to consider during splitting, balancing or consolidating TSM servers.
With TSM 5.5, restartable server-to-server export has been introduced as a feature. You can now suspend and later restart an export operation. This is a step in the right direction.

But....wouldn't it be nice to have a feature that allowed you to export just the metadata between the exporting and the importing server and than physically move the tape between the libraries? This sound too good to be true, but it actually is in the roadmap for TSM 6.1. Of course: statements of IBM plans and directions are provided for information purposes only and are subject to change without notice, but still!

The mechanism is called 'Server-Server Storage Pool Volume Transfer' and is listed as a future candidate, so beyond the initial release of TSM 6.1. It will dramatically reduce the time (and bandwidth) required for exporting nodes with loads of data. This future candidate initial functionality, at least for now, looks like it will only work for entire storagepools at a time. After the export of an entire storage pool, you can easily delete the nodes that you do not want.



In the image, and the example above, a volume is physically moved between two servers. In a shared library environment, it might be just possible to checkout/checkin or update the owner of the volume(s) to do the trick. Of course, there are some things to keep in mind here. Because a volume cannot have more than one owner, and thus, cannot have data on it from nodes spread across different TSM servers, the data of one node should be on separate, isolated volumes before the owner can be changed.

Will this require a 'move data' before the actual export? This will (at first sight) destroy the advantage that is created by this new functionality. Should collocation by node be in place before the server-server storage pool volume transfer will work?

All in all; is this, at first sight, a VERY welcome addition to TSM? Time will tell.