The tc Module

The tc ( Tokyo Cabinet ) extension command is provided by the external Tc Tcl module. It must be loaded before the command becomes available.

Example ( Tcl ):

package require Tc

or in Python

from tc import *

Tokyo Cabinet files need to be opened or created before any operations on them can commence. Afterwards, they are identified by a handle, which is returned by the opener commands. When a file are no longer in use, it should be closed.

The general syntax of the tc command follows the usual command/subcommand/handle/parameters syntax. This command is designed to be compatible to the older gdbm command, which performs the same type of operations on Gdbm files. Gdbm files generally require significantly more disk space, and are slower for larger data sets. Another big advantage or Tc versus Gdbm is that the Tc files are 32/64-bit-clean and byte-order independent and thus are more easily exchanged between systems. On the other hand, the underlying Tokyo Cabinet library is currently not supported on Windows, and therefore this command is also unavailable on that platform

At this time, only Tc hash data bases are supported, and this Tokyo Cabinet database type is implied in all commands.

Examples:

set tchandle [tc open myfile.tch]
set data [tc get $tchandle $key]
tc close $tchandle

This is the list of of subcommands:

tc add

tc add tchandle ?-nocase? key ?data?...
t.add(key=,data=,?nocase=?)

Append the listed data items as Tcl list elements to the entry identified by the key. If no such record exists, a new record with the initial set of list items is created if any data arguments are passed. If any of the data items are already present in the current value list, they are ignored, so duplicates are not added. The duplicate check is performed in case-insensitive fashion if the - nocase flag is used.

The tc append command performs a similar operation, but without a duplicate check.

The command returns the updated entry value list.

This command only works if an existing value is a properly formatted Tcl list. For Python , you can either pass a list or tuple as data argument, or a string which will be split into list items according to the Tcl syntax.

Example:

tc add $tchandle “project_$projectid” [ens get $eh E_IDENT]
tc add $tchandle “project_$projectid” [ens get $eh E_IDENT]

The second statement has no effect.

tc append

tc append tchandle key data ?data?...
t.append(key=,data=)

Append the data items as Tcl list elements to the entry identified by the key. If no such record exists, a new record with the initial set of list items is created if any data arguments are passed. No duplicate check is performed for the added list elements - this is the difference to the tc add command.

The command returns the updated entry value list.

This command only works if an existing value is a properly formatted Tcl list. For Python , you can either pass a list or tuple as data argument, or a string which will be split into list items according to the Tcl syntax.

Example:

tc append $tchandle “project_$projectid” [ens get $eh E_IDENT]
tc append $tchandle “project_$projectid” [ens get $eh E_IDENT]

The second statement adds a duplicate list element to the entry.

tc close

tc close ?tchandle?...
tc close all
t.close()
Tc.Close(?tcref/tchandle?,...)
Tc.Close(“all”)

This command closes opened Tokyo Cabinet files. After a file has been closed, its handle becomes invalid, but may the reissued for another opened Tokyo Cabinet file again.

The first version of the command closes specific files. The second version closes all open Tokyo Cabinet files. Both variants return the number of closed files as result.

Example:

tc close $tchandle

tc count

tc count tchandle ?pattern?
t.count(?pattern=?)

Count the number of keys in the file. If a pattern argument is given, only the file entries with a matching key are counted. The command returns the number of passing keys.

Unfortunately, Tokyo Cabinet files do not maintain an internal record count, so this command has to loop through all keys, which can take a substantial amount of time for large files.

The command tc size is an alias for this command.

Example:

set size [tc count $tchandle]

tc create

tc create filename ?mode? ?filemode?
Tc.Create(filename=,?mode=?,?filemode=?)

This command is the same as the command tc open , except that the default file access mode to be used if no explicit mode list is specified is create instead of read.

Example:

set tchandle [tc create thefile.tch]

tc delete

tc delete tchandle key ?ispattern?
t.delete(key=,?ispattern=?)

If no ispattern boolean argument is given, or it is not a true value, the record matching the key exactly is deleted. If the entry did exist and could be deleted, 1 is returned, 0 otherwise.

If the ispattern flag is set, the key argument is interpreted as a string pattern. All records with keys matching the pattern are deleted. In this case, the return value is the number of deleted records.

Example:

tc delete $tchandle count* 1

deletes all records with keys starting with count.

tc dump

tc dump tchandle ?file/pipe/std_channel/tcl_channel? ?keypattern? ?datapattern?
t.dump(filename=,?keypattern=?,?datapattern=?)

If no file handle is passed, or an empty string (or None for Python ) is used as file handle, this command is used to obtain the complete contents of a Tokyo Cabinet file as a dictionary. If non-empty filter patterns are specified, only those key/value pairs where the key or data part matches the respective pattern are reported.

If a file name, a Unix pipe in standard notation, a standard channel or a valid Tcl channel handle argument is specified, and the target is writable, the filtered key/value pairs are written to that channel. Every key/value pair is formatted as a simple two-element Tcl list (even in the Python interface) and written to the file. In this mode, the return value is the number of lines written.

Example:

tc dump $tchandle stdout

tc exists

tc exists tchandle key
t.exists(key)

This command returns boolean 1 if the key is in the Tokyo Cabinet file, 0 otherwise.

Example:

set isknown [tc exists $tchandle [ens get $ehandle E_HASHY]]

tc first

tc first tchandle ?pattern?
t.first(?pattern=?)

Returns the key of the first file entry. In case a pattern is specified, the first entry whose key matches the pattern is retrieved. If no key can be found, an error results.

Example:

tc first $tcandle Z*

returns the first key which starts with a (case-sensitive) Z. Keys are returned in an unpredictable order.

This command is typically used in combination with subsequent tc next commands.

tc get

tc get tchandle key ?silent? ?defaultvalue?
t.get(key=,?silent=?,?default=?)
t.key
t[key]

Retrieve the data value associated with the specified key. If the key does not exist, and a default value was specified, the default value is returned instead. Without a default value, the result is an empty string ( Tcl ) or None ( Python ) if the silent flag is set, and an error otherwise.

This command may return binary data with non-printable characters, so the language objects are byte arrays, not strings.

For Python , if the key is the same as the name of one of the class methods, only t.get() can be used to retrieve the data.

Example:

tc get $tchandle [tc first $tchandle]

tc incr

tc incr tchandle key ?delta?
t.incr(key=,?delta=?)

Increment the data value stored under the specified key by the delta value. If no delta value is specified, it defaults to one. If the key does not exist, a new key/value pair with a data value of the delta is created. If the entry exists, but the data value is not a valid integer, an error is raised. The value is stored as a string.

The return value of this command is the incremented value.

Example:

tc insert $tchandle “count_dracula” 5
puts [tc incr $tchandle “count_dracula”]

This command sequence updates the file, and outputs 6.

tc index

tc index tchandle index ?pattern?
t.index(key=,?pattern=?)

Return the nth key in the database if no pattern is specified, or the nth key which matches the pattern. The key index starts with zero.

Example:

tc index $tchandle 10 cpdname*

returns the key for the eleventh file record which starts with the string cpdname. If no such key exists, an error is raised.

tc insert

tc insert tchandle ?key value?...
tc insert tchandle dict
t.insert(?key,value?,...)
t.insert(dict)

Insert one or more new key/value pairs into the file. The values may be binary data. If the key already exists, the return value for that key/value pair is 0 and the old data is not overwritten. If the key is not in the database, 1 is returned for the item pair and the value is stored in the file.

If only a single argument is used after the handle, it is expected to be a properly formed Tcl dictionary. In that case, all dictionary elements are processed as if they were spelled out as individual key/value pairs.

The return value is a list of boolean flags indicating success or failure for the operation on each key/value pair. No error is raised if a write operation fails.

Example:

tc insert $tchandle [ens get $ehandle E_HASH] [ens get $ehandle E_GIF]
tc insert $tchandle [array get ::params]

tc keys

tc keys tchandle ?pattern?...
t.keys(?pattern=?)

This command returns a list of all keys in the file. Keys may optionally be filtered by one or more string match patterns.

The related tc match command can be used to obtain a list of keys which are filtered by the value component and not the key name.

Example:

set keylist [tc keys $tchandle project${id}*]

tc linkvar

tc linkvar tchandle ?-loadall? ?-preserve? varname

This command establishes a link between a Tokyo Cabinet file and a Tcl array variable. If the variable is already in existence, it is deleted prior to recreation as a linked array variable. This can be prevented by using the - preserve option.

The command is not supported in Python .

After the link has been formed, any read access by a Tcl script on an element of the array variable retrieves the data value from the Tokyo Cabinet file which corresponds to the array element name as key, if the variable element is not yet set. If the variable element already exists, it takes precedence over the file contents, but this can only happen if the - preserve option was used when the variable was linked. In case file retrieval is performed, and the key does not exist in the file, an error is generated.

Assigning a value to an array element replaces or creates the corresponding Tokyo Cabinet file entry. If an array element is deleted (for example, by a Tcl unset command), the corresponding key and value in the file are also deleted. Replacement and deletion operations require a Tokyo Cabinet file opened for write access.

If the -loadall option is used, all array elements are immediately loaded by looping over all Tokyo Cabinet file keys when the command is executed. By default, data is retrieved from the file only when an array element is explicitly accessed. If this option is used, the array names command immediately shows all file keys, not just the ones currently loaded.

Since Tokyo Cabinet data, once retrieved from the file by accessing the linked array element, is never deleted from memory, the use of this mechanism is not recommended for large files.

It is possible to link multiple variables to a single Tokyo Cabinet file.

Tokyo Cabinet files must not be closed when variable links to the file are active. Currently, there is no mechanism to detect that the target of a variable link as gone away, so this cannot be handled automatically. Variable links can be removed by the tc unlinkvar command.

Example:

tc linkvar $tchandle g_array
puts “Key: $thekey Value: $g_array($thekey)”
set g_array($newkey) $newvalue
unset g_array($thekey)

tc list

tc list tchandle ?pattern?
Tc.List(?pattern=?)

Get a list of currently active Tokyo Cabinet file handles or references. If no files have been opened, an empty list is returned. Optionally, a string filter pattern can be specified.

Example:

tc list

tc loop

tc loop tchandle keyvar datavar ?pattern? body
t.loop(function=,?keyvariable=?,?datavariable=?,?pattern=?)

Run a loop over all file entries. If a filter pattern is specified, only those records with a matching key are visited. After retrieving the key and data values, the key and data variables are initialized to the current key and data values, and then the commands in the body section are executed.

With Tcl , in the body, the standard break or return statements may be used to force an early exit from the loop, and the continue statement also works as expected. In case an uncaught error occurs in the body, the loop terminates with an error message.

The Python interface intentionally has a different argument sequence. The function may either be a function reference, or a multi-line string containing the loop body code similar to the Tcl style. If a function reference is used, the function is called with a key/value tuple as only argument. Loop variables are not required in this mode (though they are still supported). Normal break and continue loop control statements cannot be used in the Python functions. Instead, the custom BreakLoop and ContinueLoop exceptions can be thrown for the same effect.

If the loop was not terminated due to an error, the result value is the number of visited keys.

Example:

tc loop $tchandle key data {
	puts “Key: $key with value $data”
}

tc match

tc match tchandle pattern
t.match(pattern=)

This command returns a list of all keys where the data value matches the filter pattern. This is similar to the tc keys command, with the difference that in that command the key names are used for filtering, not the data values.

Example:

set keylist [tc match $tchandle *nitro*]

tc new

tc new filename ?mode? ?filemode?
Tc.New(filename=,?mode=?,?filemode=?)

This command is the same as the command tc open , except that the default file access mode to be used if no explicit mode list is specified is new instead of read .

Example:

set tchandle [tc new thefile.tch]

tc next

tc next tchandle key ?pattern?
t.next(?pattern=?)

Get the next key after the specified key which matches the pattern. If no pattern is specified, no key filtering is performed. If no key can be found, an error is raised, otherwise the key is returned as command result.

A starting point for a key traversal is usually obtained via a tc first or tc index command.

Example:

tc next $tchandle name20 name*

produces the next key after the current key name20 matching the pattern name*. This key is not necessarily name21 , or any other predictable value. Rather, the key sequence in the database is determined by its internal hash, and the ordering is pseudo-random. The only guarantee is that with a tc first/tc next loop all keys are ultimately visited. A deletion of the current key is allowed and guaranteed to not disturb the traversal sequence.

tc open

tc open filename ?mode? ?filemode?
Tc(filename=,?mode=?,?filemode=?)
Tc.Open(ilename=,?mode=?,?filemode=?)

Open an existing Tokyo Cabinet file, or create a new one. The command returns a new object handle or reference. The default mode is read , opening the file for reading. In this case, the file must already exist. The mode parameter may consist of a collection of the following keywords:

In standard applications, the mode list is either a single word describing the access mode, or a list of the fast or compressed attributes and the access mode.

The optional filemode parameter determines the Unix-style octal file access bits which have an effect only in case the file is created.

This command returns a Tokyo Cabinet file handle in the form tc%d which can be used in subsequent tc commands, until the file is closed and the handle becomes invalid.

tc ref

Tc.Ref(identifier)

Python -only method to get a reference of the Tokyo Cabinet object from its handle.

tc reorganize

tc reorganize tchandle
t.reorganize()

Compact the data file. This can be useful after many deletions. This command requires write access to the file.

tc reorg is an alias.

For large files, this command can take a long time to complete.

Example:

tc reorg $tchandle

tc replace

tc replace tchandle ?key value?...
tc replace tchandle dict
t.replace(?key,value?,...)
t.replace(dict)
t.key = value
t[key] = value

Store one or more specified values which may be binary data, under the given keys. If a key is already in use, the old value is overwritten. If only a single argument is used after the handle, it is expected to be a properly formed Tcl dictionary. In that case, all dictionary elements are processed as if they were spelled out as individual key/value pairs.

The return value is a list of boolean flags indicating success or failure for the operation on each key/value pair. No error is raised if a write failed.

tc set is an alias for this command.

Examples:

tc replace $tchandle $thekey $newdata $key2 $moredata
tc replace $tchandle [array get ::params]

tc restore

tc restore tchandle filename ?callback?
t.restore(filename=,?function=?)

This command reads a text file which usually was produced by the tc dump command and adds its contents to the current Tokyo Cabinet file. Every line of the file is expected to contain a properly formatted two-element Tcl list. The list is split into the key and data parts and the contents are written to the Tokyo Cabinet file, replacing old entries in case of existing keys.

The input file may be gzip-compressed or plain ASCII text.

If a callback function is specified, it is called with the object handle (or reference), the input file name, and the current restored item count after each successful line input.

The command tc readfile is an alias for this command.

The return value is the number of file lines read.

Example:

tc restore $tchandle “totally_superfluous_backup.txt”

tc set

This is an alias for tc replace.

tc sync

tc sync tchandle
t.sync()

Synchronize the memory and disk status of the file. All pending changes are committed to disk. This command has an effect only if the file was opened in fast mode (see tc open command).

tc synchronize is an alias.

Example:

tc sync $tchandle

tc unlinkvar

tc unlinkvar tchandle ?-preserve? varname

Unlink a Tcl array variable from a Tokyo Cabinet file. If the - preserve option is not used, it is also deleted from the Tcl interpreter. The association between array variable and Tokyo Cabinet file is also automatically broken when the variable is deleted by other means, such as a Tcl unset command.

This command is not supported in the Python interface.

The Tokyo Cabinet file contents are not modified by unlinking by means of executing this command or any other method of variable deletion.

Variable links are created with the tc linkvar command.

Example:

tc unlink $tchandle g_array