Hume Datahub SDK Recent Changes
This document was last changed on $Date: 2018/02/13 17:53:50 $.
Changes are made to this document to inform the user
community of recent changes to the Datahub SDK toolset
software. This document does not contain most of the
detailed changes that are described in the changes document of the
previous SDK version.
SECS/GEM Software Changes
- Customers who build custom versions of the SecsServer.exe program
are advised that the file system paths contained within the SecsServer
program file have been revised. New custom versions should be
built using the newer Hume provided SecsServer.exe file as the starting
point, and care should be taken not to mix older and new version
building scripts. (11/21/17).
- The logic for periodic "file saving active" messages when logging
SECS communication has been revised to fix an issue with Daylight
Savings Time changes. (3/20/17).
- The Conversation feature of the Tcl/Tk applications has been updated
to guard against overly long messages being displayed by the Validate
message dialog. (8/1/16).
- The Trace windows and Status windows of the Tcl/Tk SECS apps have
been enhanced to simplify copy-and-paste usage of the SECS communication
- Environment variables DMH_CONNECT_DEBUG, DMH_CONNECT_WAIT, and DMH_CONNECT_RETRY
may be defined and set to override the default timing during startup
when a .NET, C++, or Java app connects to the newly started SECS Server
background process. See the .NET SecsPort DebugEvent or the ServerError event documentation for details. (1/6/16).
- A startup problem with paths containing white space and use of the
SECS Server on Windows was fixed for the C++ equipment and host
- The HSMS connection management logic was improved to retry and
recover certain socket exceptions that formerly resulted in a
- The installation of the Datahub SDK on Linux systems has been
revised so that if the system is installed by root no further setting of
file or directory permissions is needed to enable persistence for
ordinary users. (7/21/14).
- Changes were made to the .NET HumeDMH.dll and to the Tracewin
classes to support receiving and sending ASCII data (TSN type A) that
embeds null characters. Similar changes have been made to the Java
libraries and the VB6 libraries to enable using null characters in ASCII data. The C++
and Tcl libraries use the modified UTF-8 byte sequence, 0xc0 0x80, to
represent null characters which enables their use in ASCII SECS message
- The directory used for Stream 13 dataset transfers can be
customized by setting the (DatasetXferDir) value in the SECS Server
connection array during initialization. (4/23/14).
- The Hume Java applications have been updated to use JDK 1.7
features. Hume also provides older versions that are compiled for
JDK 1.6 and 1.5 on the download site. Only the latest versions are receiving new feature additions. (1/9/14).
- Diagnostic messages for HSMS socket exceptions and closes have
been improved to better indicate which side initiated the close and to
more clearly indicate when additional connection attempts are
- The .NET equipment and host libraries were revised to have better
GUI performance of the Tracewin class. There is noticeably
better GUI responsiveness when large Stream 13 transfers are being
displayed in the Trace window. (8/21/13).
- A new feature of being able to receive or send Stream 7 messages
direct from files has been added. This feature makes it possible
to work with S7F3 or S7F6 Process Program transfers that are as large as
16 meg without incurring large memory allocations. To access the
new feature, the HSMS connection array value ppbodyFileMode is
set to 1. This feature has been added to the .NET, Java, and C++
libraries as a new configurable property value. (5/25/13)
- A new property, TraceLogMaxWidth, has been added to most of
the libraries to provide a means of reducing the size of log files when
the large messages are used. The data of a message body is logged in a single line.
You are able to configure a maximum number of characters per output
line. Lines that are in excess of the maximum are trimmed and a
short explanation is appended. (5/7/13).
- The .NET documentation now suggests adding the SecsPort (or SecsHost)
projects to your application solution instead of only adding References
in order to receive better prompting when editing code. The source
code was edited to include more prompting text. (8/16/12).
- A default maximum message size for HSMS is now being set at
2,048,000 bytes. Previously the default behavior was to have no
limit; a limit was only imposed if the recv_max
connection array element was set. This change makes the
software resilient to buggy software that sends corrupted message
length data in the HSMS packet headers without the developer needing to
configure a maximum size. (3/2/12).
- The Tcl/Tk Conversations feature has been enhanced with a new
HTML Report feature, resizing of editor windows, expanded size limits,
execution of pre-send code when validating message data, and improved
merging when loading. There are new files of host Conversation
items which are optionally loaded to exercise E39 Object, E40 Process
Job, E87 Carrier, E90 Substrate, E94 Control Job, and E116 Performance
messages. (8/20/11 and 1/16/12).
- TSN notation now accepts decimal integer values with I8 and U8 64
bit integer values and provides decimal integer values for output.
- The Tcl Host Supervisor PROC_STATE_NAMES Conversation item was
revised to fix a minor bug with displaying the correct initial process
state for equipment that deploys the ProcessStateNames variable.
- The Tcl Host Supervisor has new 300mm Conversation Items that are
loadable from the ../gem2/custom directory that provide for defining
event reports and enum variable value mappings for the E40, E87, E90,
and E94 300mm standards. The Conversation Items in 300mm_host.cvt
can be loaded and the first item, 300mm-1, can be "Sent" as a shortcut
to load and "Send" several
Conversation Items. We show "Send" in quotes because the Send
action is used to execute the Tcl code in these Items but SECS
messages are not always sent. "Sending" the 300mm-UI item creates a
User Interface for exercising 300mm features including working with
Ports, Carriers, Process Jobs, and Control Jobs. (12/12/17 and 1/20/18).
- Handling of alarm information during initialization was revised to
synchronize the set/clear state using S5F8 reply data. Also, S5F5R
is sent or not sent per the initialization configuration whereas
previously there was optimizing logic that would conditionally skip
sending the message. (12/8/17).
- The Tcl Host Supervisor supports adding a variable EVENT_DESCRIPTION
to event reports with the value being set by the host logic using event
descriptions known from the equipment variable EventDescriptions or
configured. Also, a new checkbox provides for adding the formatted
event reports seen in the Recent Reports list to the SECS trace for
enhanced viewing or logging. (11/21/17).
- The ei_event_log table schema has been updated. The length of
data column was changed to 60000 from 20000 which allows for processing
larger event reports in this table. Received event report data is
parsed and individual variable values are saved in the ei_variable
table, so the limits of the ei_event_log table only affects certain
modes of processing event report data such as the display to the Tcl
Host Supervisor UI. New error catching logic has been
added in case the length limit is exceeded. (12/19/16).
- New Conversation items were added to the Tcl/Tk Host Supervisor for
interactive use of S1F21R (DVVALS_QUERY) and S1F23R
(EVENTS_QUERY). The latter item can be used to create Report
Definitions for the Event DVVAL associations. (12/14/15).
- The initialization for Stream 13 usage was revised to fix a problem with initializing multiple connections. (11/26/15).
- A minor fix was made to the Tcl Host Supervisor to correct handling
of failure replies to S1F21 or S1F23 during initialization. (6/18/15).
- The .NET, Java, and Tcl host GUI applications have an added feature
of optionally configuring RPTID values for event reports. This
simplifies working with less capable equipment that cannot use CEID
values for RPTID values. It also reduces the effort needed to
manually configure sets of similar event reports. (1/24/14).
- The Tcl Supervisor application was revised so that the ei_startup
table records are interchangeable with .NET and Java, host supervisor
applications. Also, more options have been made configurable from
the Tcl connection startup dialog including new message types S1F21 and S1F23. The ei_startup table schema was
revised to include new fields so host developers will need to use
updated versions of the SecsServer with updated versions of the Datahub
- The Tcl secs_xact procedure was revised so that re-entrant execution by Tcl applications is possible.
Also, the asynchronous initialization flow was tweaked so that user
written custom initialization procedures are executed from the event
loop using after instead of from a SECS receiving callback. (8/8/13).
- A fix was made for the error reply of SendSecsMsg (Java or .NET) when communication is disabled (8/20/12).
- A change was made to prevent extra attempts at communication if
communication is enabled repeatedly. The extra attempts also
occurred if communication was disabled momentarily and
re-enabled. The extra attempts were only an issue when
communication failed. (7/25/12).
- New asynchronous initialization logic that does not nest the SECS
Server execution stack has been introduced for the benefit of host
applications that connect to large numbers of equipment.
(2/28/12) and (12/22/12) A fix was made in the use of S2F29 on
- Support for Stream 7 recipe transfer for recipe body data of type
U1 was added. (2/8/12)
- A new Host C++
class library and demonstration application is available for
Windows and Linux. (12/1/11).
- A .NET Visual Basic program is newly available to demonstrate
using the SecsHost Component from VB. The SecsHostEditor class
has been moved from the GemHostApp C# project to the SecsHost component
so that it can be used from a VB application. (10/26/11).
- The variable discovery logic was tweaked to fix the names of
items that are discovered by event reports occurring before the
variables are queried and discovered (1/30/12).
- The saving logic for uploaded recipes has a new feature to allow
sharing uploaded process programs with multiple equipment
connections. If the array element recipes_are_shared is defined and
set to 1, the recipe description records are made shareable. Each
interface should be configured to use the same recipe upload directory
when the recipes_are_shared property is defined and set to 1, and the
equipment connections should execute in the same SECS Server process.
- The logic for handling S2F15 messages that set the value of ECV
Parameters has been updated. If a numeric value type is specified
without a size indication (eg., I2 and not I2:1), and the ECMIN and
ECMAX values are empty strings, then the host is allowed to set an empty
value. If the host sets a non-empty numeric value, it should be a
scalar value (size 1) when there is not a size specified, otherwise the
received vector size must match the specified size. In the usual
situation where the host is setting numeric type values, either the
ECMIN and ECMAX values may be empty strings in which case the
corresponding check against a min or max value is skipped.
However, the passed numeric data value must be within the possible range
of the configured value type. (6/9/17).
- The example Tool UI Tcl application has been updated to avoid
replacing customized versions of the custom online procedure and the
operator spooling alert procedures. This surfaced as an issue when
using Tcl OPC integration. The startup logic in gemsim was
changed so that the Tool UI sets a different value of MODLN, the model
- Methods to send HSMS linktest messages have been added to the C++,
.NET, and Java APIs. The Tcl secs_xact procedure has been enhanced
to handle linktests too. An example use would be to force
discovery that the SECS communication is up or down just before an
important event report. Said another way, polling with linktest
messages can be used to force discovery of a broken connection, so that
spooling is activated sooner or so that the app can synchronize with the
link state. (1/27/16).
- When the host sets ECV Parameters using S2F15, the t_latest field in
the ei_variable table row is now updated to enable notifications
whenever the host sets the same variable value. An interested
client can open a subscription to changes in the t_latest field in
addition to the usual subscription on variable value updates.
- Additional error trapping was added to the event posting logic in
order to provide diagnostic messages for improper variable values.
- The maximum length of an EventName was raised to 256 from 32. (2/23/15).
- The property PPFilesByRequest and a file request event have been
added to the .NET, Java and C++ libraries. In March 2015, transfer
logic was revised to allow using process program names that are not
valid filenames when using the FilesByRequest feature. (11/7/14) and
- DATAID value logic has been revised such that an incrementing value
can be sent instead of the CEID value if the application sets the
connection array element (dataid) to a starting unsigned integer value.
- The reply logic for S2F23, Trace Initialize Send, has been revised
to use newly added TIAACK codes 4 and 5 instead of S9F7. (6/20/14).
- New methods have been added to provide a Status Variable, ProcessStateNames,
that describes the process states. This variable is not required
by the standards but is recommended by Hume. The Tcl/Tk Host
Supervisor application recognizes this variable and uses it to enhance
the display of Process States. (2/27/14).
- A new method, serverLicenseCheck, can validate the runtime license when using the SecsServer[.exe] program. (2/25/14).
- The logic for adding ECV Parameters has been revised to feature better validation of the method call arguments. (2/21/14).
- A new property, SpoolDirectory, has been added to the C++, .NET, and
Java libraries to provide explicit control over the file system
directory used for persistence. The updated libraries require
updated versions of the SecsServer or Datahub SDK to set the directory
- Persistence of the reporting configuration has been revised to save
after no changes have been made for one minute instead of only at
program exit. (11/11/13).
- HSMS logic has been updated so that applications cannot send SECS
data messages unless the connection is in the selected state. This
has surfaced as an issue for host software that is slow to issue the
Select.Req control message when the connection is made. (11/11/13).
- .NET variable value logic had a flaw that affected the less common
situation of using vector variable values with VarValueRequest callbacks
and posting events from a non-GUI thread. The flaw was fixed by
revising files in the SECS server; there was no update of the SecsPort
- Handling of S1F13 was changed so that redundant messages do not cause additional Control State event reports. (4/25/13).
- Handling of S2F37 and S6F23 was improved to allow type B values for
integers and to reply with S9F7 for more cases of improper values.
S7F1 handling was improved to strictly validate PPID as a valid
filename and to perform a maximum length check. (4/18/13)
- Hume has available to SDK purchasers a documentation generation
application that creates a customizable SECS/GEM Reference Document from
your running equipment interface. This application is not on the
download site. The application supercedes editing a template
document using the output of the ei_html_save procedure. (1/1/13).
- The limit on the number of variables in an event report was raised
from approximately 200 to over 1000 by revising the ei_report table
- A new example application featuring a simple
production SECS/GEM UI has been added to the .NET Equipment
distribution, the Windows C++ (SecsEquipCpp) distribution, and the Java
SecsEquip distribution. A Start Menu Program item has been added
Tcl/Tk Tool UI program. (10/4/12).
- New methods varGroupSet and varGroupSetEventPost have been added to
the .NET, C++, and Java equipment libraries. The methods provide
higher performance and data integrity for applications which manage SECS
variable values in SECS Server variable table. The Java library
was updated to feature group value requests when variable value
callbacks are used. This technique is already present in the .NET
and C++ libraries. (9/14/12)
- The SECS server logic for replying to S1F4 was revised to improve
performance when using more than 600 variables whose values are provided
by .NET, C++, or Java equipment applications. The .NET Equipment application
was updated to include an example of using 1000 Status Variables.
- Methods StartupLoad, StartupRestore, and StartupSave have been
added to the VB6 SecsEquip. A new example tool UI and
configuration dialogs have been added to the project, but they have not
been integrated into the example application screen flow. They can
be used as a parts kit by the tool developer. (9/1/12)
- The Tcl/Tk SECS Server software can show a simple tool operator user interface window that can be used directly by
equipment. To see the UI, run the gemsim program in the gem2/equip
directory with "set TOOL 1" as a command line argument. Right
click on the Communication status LED to bring up the Administrator's menu which allows terminating the program. (8/11/12).
- Customization features have been added to the Tcl gemsim application to support direct use for a custom deployment instead of needing to copy and edit it. (8/11/12).
- New features have been added to make it easier to deploy an
equipment SECS interface using only OPC data items. (6/14/12).
- Length limits for MDLN and SOFTREV data items have been raised to
20 from 6 in accordance with the E5 revision. (3/26/12).
- A new Built-in variable, DefaultReportID,
and a new feature of providing a default Event Report for every Data
Collection Event type that does not have any associated reports have
been added. The Hume Host Supervisor program has a new
Conversation Item to utilize this optional feature. There are
brief usage notes in the Equipment
Builtins document. (1/19/12).
- Built-in support has been added for the S1F21, S1F22, S1F23, and
S1F24 discovery message types presented in standard ballot 4824B.
A new method, EventDvvals(), has been added to the .NET, Java, and C++
equipment libraries so that the application can specify the DVVAL
variables that are associated with application added data collection
- A SECS server procedure, ei_html_save,
has been added for the
of saving the Alarm, Event, and Variable definitions for a particular
equipment interface in an HTML file that is formatted for end
users. The procedure helps with creating SECS reference
documentation. The procedure was enhanced to include DVVALs for
each event type and to not require any arguments for the typical
deployment of only one equipment interface. The Tcl Simulator
has a new button action on the Variables pane to invoke the
procedure. The .NET SecsPort provides a method HtmlSave to call this procedure. (8/5/11) and (8/15/12).
- A new C++ Equipment library, SecsEquipCpp, is now available for
Windows and Linux in both 64 bit and 32 bit versions. The
library takes advantage of std::string and STL classes for return types
with automatic memory management. The older VCSecsEquip remains
available for Visual Studio 6. Any C++ developers using Visual
Studio 2005 or newer are encouraged to migrate to the superior new
version. See the library
documentation and the version difference notes at the end.
We are pleased to inform you that support for OPC is part of the
Datahub SDK version for Windows. There is a Tcl command ::opc::opc and a high-level
application, the OPC Supervisor.
There are also example procedures for the OEM to easily deploy an
equipment SECS interface with a controller system that uses OPC
- The OPC Supervisor app has newly-added configurable limits when
using the Browse or Discover features to enable working with servers
that have very large numbers of items. Also, the Browse windows have
a new menu action to discovery Items underneath a selected branch
of the tree. (10/23/17).
- The OPC Supervisor application was updated to disconnect client
connections when exit is called so that the application exits cleanly.
- The OPC client software is being successfully deployed by
SECS/GEM equipment OEMs. New features have been added to directly
use OPC data items to manage displaying SECS Server windows including a
configuration dialog. An improvement has been made to handle
version 2 discovery finding new items but ending before completion with
an error result. (6/29/12).
- The conversion of received data timestamps to the local timezone is
now done in a catch in case the data is not valid. (11/5/15).
Tcl/Tk Application and API Changes
- A fix was made to window placement logic in the Tk procedures
dmh_gets and dmh_win_shrink to fix an issue with dual monitors on
Windows when the right monitor is defined as the main screen. (2/13/18).
- The unzip.exe program included with the Windows SDK
installer was updated to version 6.0 after a report that the older
version failed to run during an install on Windows 10. (10/13/17).
- The dmh85 (dmh_wish) executables on Unix now accept the command line option -console
with different behavior. This option now provides a console
window using Tk which replaces use of the terminal shell. The Tk
console features use of the cursor for command history recall as well as
consolidated input and output. Prior to this change, use of the -console
option would enable terminal use when there was an input file and
terminal use would ordinarily be disabled. This feature was revised in
2017 so that when -notk is specified with -console on POSIX systems,
X-Windows is not used; instead the terminal window is enabled for
console IO. (9/15/14) and (6/30/2017).
- The Hume SDK license check logic has been modified to be case
insensive when matching the system hostname to a dmh_license record. (5/14/14).
- The Tcl/Tk SECS/GEM applications were revised to inform the user of possible licensing issues at startup. (3/25/14).
- The Datahub SDK Setup program was revised to retry and trap errors
with adding Program Items to the Start Menu. A failed
installation was reported where success was obtained after running the
setup program a second time. (11/14/13).
- The Windows regset -announce command was revised to use
SendMessageTimeout instead of SendMessage so that it can be used on
systems that are running non-responsive applications. The issue
surfaced as the cause of a failed Datahub SDK installation. (11/14/13).
- The .NET DmhClient component library, HumeDMH.dll, had a
synchronization flaw that caused rare spurious timeouts for local
DoXact calls on very fast systems. (5/10/2012).
- The Hume udp command was revised for IPv6 compatibility. (2/1/12).
- Linux Datahub SDK distributions now include Desktop menu program
items which are optionally installed using the SETUP program.
- The Tcl/Tk Datahub has a new set/query option, ADD_SEMICOLON,
which causes a semicolon to be appended to SQL statements created by
subscriptions or when saving table data. Also, the SQL insert
command has been updated so that the list of column names is
optional. These two changes ease interoperability with the
persistent database Sqlite3. (9/25/10).
Tcl/Tk 8.5 Compatibility Changes
- The current release of the Hume Datahub SDK includes Tcl/Tk
are based on Tcl & Tk patch level 8.5.16 as of September, 2014.
Prior to this, the DMH85 SDK was based on patch level 8.5.15 from December
2013. From a compatibility
point of view, Tcl 8.5 differences from Tcl 8.4 are minor and present
few problems (more
- The Tcl 8.5 distribution is not compatible with the open source
tbcload package. The Hume Data Collection Component (DCC) command
dc_encode which is used to obscure passwords is now a compiled dmh
package command and the DCC no longer requires tbcload.
With this change, the dmh package version has been changed to 1.3 from
- Since Tcl version 8.4, the Hume dmh package Tcl extension has
been revised to use
the ::dmh namespace to prevent name conflicts such as occurred
with the Tcl 8.4 lset command.
Read more about the new Tcl 8.4 namespace and compatibility issues in
document. The Hume provided Tcl commands dmh_import, dmh_var,
and vset were added to the 8.3 (DMH83)
distribution and newer Tcl/Tk distributions to support
deploying software that is compatible with Tcl 8.3 and newer.
- The Datahub SDK Tcl/Tk 8.5 distribution includes updated versions
the optional packages [incr Tcl], [incr Tk], and Iwidgets.
You need to use the versions that are matched to your version of
Tcl/Tk. Hume provides a modified index file for these packages so
that the correct version is chosen when multiple versions are
New with Tcl/Tk 8.5
Hume Changes to the 8.5 Tcl/Tk SourceForge Distribution
- Hume has diagnosed and fixed a problem with executing wish -help
(Sourceforge bug report 1938580). (4/9/2008)
- Hume has fixed a problem in the Tk library tk::EntryAutoScan
which affected ordering keystroke entry on Windows, when using entry
with modal windows, and having done a mouse drag selection of text in
entry widget, and releasing the mouse button to the left of the entry
This bug and fix have been reported using SourceForge. (12/11/01)
- The command memory_stats
is added, and compiled by default for Windows.
- The windows console menu has a new item to display
errorInfo, and a new item to display the HTML documentation. Bindings
are added for Motif style
cut-and-paste and Windows style right button paste. These changes are
done at runtime without changing the core code. The changes that
Hume has made in the past to make the console safe for memory use are
no longer needed - the core has similar logic. However, Hume has
revised the console trimming logic for better performance, and has
added a "fast mode" which optionally shortens long output lines.
- The Tk C code is patched for Windows so that the send command and
winfo interps work using the Hume DDE command.
- The Tcl library is patched so that index files of the name tclindex.txt
are used if tclIndex is not found. This enables Tcl to execute
an ISO 9660 CDROM image where tclIndex is not a valid filename.
- The Tk C code is patched to create an option to the winfo
Executing winfo timelastevent returns a value that changes
each new window event at your display. This value will be unchanged
time when there has been no user interaction with the window interface.
An application can poll this value using after, and determine if the
has been inactive. Users should migrate to the new Tk 8.5 command
inactive for this purpose - winfo timelastevent will not be patched
into the next version.
- Hume compiles the Tcl DLL on Windows so that 256 socket
are possible. The default compilation results in a limit of 64.
- The windows tclWinPipe.c file is patched so that built in
like "type" work without the user specifying cmd.exe /c. Also, windows
files with the extension .cmd are made executable. Hume has
also revised handling of windows interactive shell commands in the Tcl
init.tcl file to allow mixed case commands.
- Hume converts the Tcl/Tk man pages to HTML, adds new pages, and
custom indexes. Hume has revised the encoding
man page which had erroneous information regarding Tcl's internal