Porting from Toolkit 1.0 to 1.1
This documents describes the differences between the SILC Toolkit 1.0 and
1.1 and should help application programmers to port their SILC applications
to the new Toolkit version.
General
Platform changes
Client library
Utility library
Application utility library
Crypto library
Key Repository library
VCard library
HTTP library
ASN.1 library
General changes
The main SILC Toolkit header file has changed its name from silcincludes.h
to silc.h. The first task in porting from 1.0 to 1.1 is to change these
filenames in your source tree.
Platform changes
The Windows support has been made better by improving the SILC scheduler
and network routines. Calling silc_net_win32_init and silc_net_win32_uninit
is not needed anymore. The network routines on Windows are enabled
automatically.
Client library, lib/silcclient/
The client library in Toolkit 1.1 has been partially rewritten. It was
rewritten to work out some technical issues and to increase performance
and decrease memory consumption. The client library API has also faced
some changes, altough most of it has remained the same. Most major change
in the client library is that is now fully supports multiple threads. In
the past making multithreaded SILC client application was always a bit hard
but with Toolkit 1.1 it should be relatively straightforward.
SilcClientOperations structure
The SilcClientOperations structure has changed significantly. It no longer
has `connected', `disconnected' and `failure' function pointers. Instead a
new SilcClientConnectCallback has been defined that is given as argument to
functions like silc_client_connect_to_server, silc_client_connect_to_client
and silc_client_key_exchange (last two being entirely new functions in the
API). That callback is called when connection is established and when it
is disconnected, or if a protocol failure occurred during connecting. That
callback also delivers a detailed error information when error occurs. New
connection related status types in SilcClientConnectionStatus has also been
added, see the API for the details.
Also the `detach' client operation function pointer has been removed. Instead
the detachment data is now simply delivered in the SILC_COMMAND_DETACH
command reply.
Some of the remaining client operation function pointers have changed a bit.
Most major change is the `command' operation now having the arguments that
user gave on the user interface sent back to application. The `command_reply'
does not anymore include the SilcCommandPayload as an argument. Also the
variable argument lis is now delivered as va_list. Also the boolean
success indicator is removed and SilcStatus error argument instead is telling
the exact error if the SilcStatus status is other than SILC_STATUS_OK.
The `get_auth_method' client operation now includes the authentication method
that has been resolved from the remote server. The application no longer
needs to resolve it and the function silc_client_request_authentication_method
has been removed from the API.
The `verify_public_key' client operation now include the SilcPublicKey pointer
instead of the public key in raw encoded form. Also the connection type
argument has changed to SilcConnectionType.
The `key_agreement' client operation now includes a new protocol argument
which tells whether the key agremeent is supposed to be done on TCP or on
UDP transport. The new Toolkit 1.1 supports both.
SilcClient
The SilcClientParams includes new fields and some fields have been removed
or moved to other places. All connection timeout related fields have been
moved to SilcClientConnectionParams. The nickname_parse callback pointer has
also been removed from SilcClientParams. The Toolkit 1.1 now automatically
parses formatted nicknames.
The new field boolean `threads' can be used to tell whether the new SilcClient
is to use multiple threads or not. If threads support has been compiled in
and that field is set to TRUE then the client library will create new thread
for each new connection to a remote host. If you are going to use several
connections or some of the connections are high throughput connections (like
multimedia connections) then using threads is recommended for performance.
Note that your application needs to be multithreaded and, if necessary to,
perform concurrencly control in client operation callbacks which may be
called from multiple threads at the same time for different connections.
However, it is always guaranteed that for one connection the client
operations cannot be called from multiple threads. The multithreads
support in Toolkit 1.1 client library means that each new connection and
everything related to that connection is handled in own dedicated thread.
The silc_client_init function now takes several new arguments. It now takes
the username, hostname and realname as argument that used to be required
by the application to update directly to SilcClient pointer. This is not
longer required. Also the nickname is no longer required to be put to
SilcClient context as it has been made SilcClientConnection specific.
The function also takes a SilcClientRunning callback as argument which will
be called when the client library is running. Only after the callback is
delivered may the application start using other client library API functions.
For example, connecting to a server will fail if it is performed before
the SilcClientRunning callback is called. The callback is called after the
silc_client_run or silc_client_run_one has been called.
The silc_client_stop function takes new SilcClientStopped callback as
argument, and it will be called after the client library has been completely
stopped. The application should not quit before that is called in order
to clean up all resources allocation by the client library.
It is also not required to put the public key pair (SilcPublicKey and
SilcPrivateKey) into SilcClient. They have been made connection specific so
you simply give them as argument when you create new connection. This way
it is now possible to use different key pairs with different connections,
one that was not possible in Toolkit 1.0 without creating new SilcClient
instance for each new connection.
SilcClientConnection
The SilcClientConnection represents connection to a remote host. In new
Toolkit 1.1 this structure now includes all connection related data and
no connection related data is anymore included in SilcClient. For this
reason it is no longer necessary to create multiple SilcClient instances
just because you want to create multiple connections.
The SilcClientConnectionParams structure has many new fields. This structure
is given as argument to any function that is able to create a new connection
or to create a network listener or to create a new SILC sessions. See the
API documentation for all the details but the following one field of special
relevance when creating new SILC server connections. The nickname field is
now included in this structure and it is the nickname user would like to
initially use in the SILC network.
The functions silc_client_connect_to_server, silc_client_connect_to_client
and silc_client_key_exchange can all be used to create new SILC session with
a remote. To connect SILC server use silc_client_connect_to_server. You
now give the SilcClientConnectionParams as argument which includes the
nicknameuser wants to use, you now can give the public key pair as argument that is
to be used in the connecting. The new SilcClientConnectCallback will be
called once the connection has been established and it delivers the
SilcClientConection context to the application. It will be later called
again to indicated disconnection from the remote host. These functions
now return SilcAsyncOperation context which can be used to cancel the
connecting, if necessary. In the past canceling connection was not possible.
SilcAsyncOperation op;
SilcClientConnectionParams params;
memset(¶ms, 0, sizeof(params));
/* Set nickname user wants to use */
params.nickname = nickname;
/* Set perfect forward secery for key exchange */
params.pfs = TRUE;
/* Example of creating, and then canceling a connection */
op = silc_client_connect_to_server(client, params, public_key, private_key,
remote_host, remote_port, connection_cb,
app_context);
if (!op) {
fatal("Connecting failed immediately"));
exit(1);
}
...
/* Cancel connecting. After canceling the `connection_cb' will not be
called. */
silc_async_abort(op);
The old function silc_client_start_key_exchange has been replaced with
silc_client_key_exchange function. Semantically it is equivalent to
silc_client_connect_to_server but it does not create new connection. Instead,
it accepts SilcStream as argument which is already established connection
to a remote host and it merely starts the key exchange. See an example
in the API documentation on how to use the silc_client_key_exchange, if
your application wishes itself to create connection instead of using the
client library to do it.
The functions silc_client_add_connection, silc_client_del_connection and
silc_client_del_socket has been removed. They are no longer needed.
Entries
Just like in Toolkit 1.0 we now have SilcClientEntry to represent user,
SilcChannelEntry to represent channel and SilcServerEntry to represent
server. In the past these structures and all API functions that dealt
with them were in silcclient.h file. They
are now in silcclient_entry.h in
Toolkit 1.1.
As an general convention each of these new entries now are reference
counted and they have locks when using them in multithreaded application.
Even in one-threaded application the application must always acquire a
reference of the entry if it wishes to save the entry pointer in the
application. The reference must be released once the entry pointer is
not needed anymore. This ensures that the library cannot free the entry
pointer underneath the application. An own API for taking and releasing
the reference is in Toolkit 1.1.
/* Take reference of the client entry I save to my own context. */
my_context->client_entry = silc_client_ref_client(client, conn, client_entry);
If multiple threads are used and application wants to access the entry it
must always first lock the entry. After application has read the information
it needs from the entry it must release the entry lock. The entry lock
should be held only for short periods of time and failure to release the
lock will result into deadlock. An own API for taking and releasing the
entry lock is in Toolkit 1.1. If you application is not multithreaded
you do not need to use the entry locking.
/* Read data from client entry in multithreaded environment */
silc_client_lock_client(client_entry);
fprintf(stdout, "%s\n", client_entry->nickname);
fprintf(stdout, "%s\n", silc_id_render(SILC_ID_CLIENT, &client_entry->id));
silc_client_unlock_client(client_entry);
From the entries all unnecessary information for application has been either
removed or moved to internal structure that is not accessible by the
application. As a result the entry structures are much smaller and cleaner.
The SilcClientEntry now includes nickname_normalized field and it is the
normalized version of the user's nickname, in case application needs it.
It is also guaranteed in Toolkit 1.1 that the nickname inside SilcCientEntry
is always valid nickname.
The SilcChannelEntry now includes new channel_pubkeys list, which includes
the channel public keys if they have added to the channel. This information
was not present in Toolkit 1.0.
The SilcServerEntry now includes new field public_key, which is the server's
public key if we have resolved it. This information was not present in
Toolkit 1.0.
In Toolkit 1.1 it is now easier to search and resolve entries. As a new
feature it now also possible to search and resolve server entries from the
SILC network. See the API documentation for all the details on searching
entries from the client library cache and from the SILC network.
Sending/Receiving messages
Sending messages has not much changed from the Toolkit 1.1. In Toolkit 1.1
the message may now safely be sent in from multiple threads for same client
or for same channel. In case the message are digitally signed the hash
function used in the signature computation must now be given as argument.
If this is done in multiple threads, each thread must use different SilcHash
context because SilcHash does not support multiple threads. Simply allocate
new SilcHash for each thread where you send digitally signed messages.
Receiving message is same as in Toolkit 1.0. You receive private_message
or channel_message client operation. It is always guaranteed that even in
multithreaded application the messages are received in one thread. You need
concurrency control in your application only if you access shared data in
your client operation callbacks.
In Toolkit 1.1 as a new feature it is also possible to wait for incoming
private messages in a thread. New function silc_client_private_message_wait
can be used to block the calling process or thread until private message
for the specified client is received.
Calling and sending commands
Just like in Toolkit 1.0 in Toolkit 1.1 you can call command implemented
inside the client library with silc_client_command_call. The command_reply
client operation will be called once the command reply has arrived.
As a major change in semantics of sending commands with the
silc_client_command_send function is the way the command reply is handled
in Toolkit 1.1. In the new Toolkit the command_reply client operation will
not be anymore called for commands that has been sent with
silc_client_command_send. The command_reply client operation is called only
when silc_client_command_call function is used. With silc_client_command_send
you can give the command reply callback, SilcClientCommandReply, as argument,
and it will be called for each command reply that is received from the
server.
Just likein 1.0 in 1.1 it is also possible to attach to pending commands
by using silc_client_command_pending. As a difference to 1.0 the command
identifier is not anymore available to application from the
SilcClientConnection context. Instead the silc_client_command_call and
silc_client_command_send return the command identifier, and application needs
to save it in order to be able to attach to it at later time. However,
this feature is not expected to be very important for application
programmers as the new silc_client_command_send already includes the
command reply callback.
Comparison between 1.0 and 1.1
Toolkit 1.0:
/* Send ping command. The reply will be in the attached command reply
and in the command_reply client operation. */
silc_client_command_send(client, conn, SILC_COMMAND_PING, ++conn->cmd_ident,
1, 1, server_id, server_id_len);
silc_client_command_pending(conn, SILC_COMMAND_PING, conn->cmd_ident,
ping_command_reply, context);
Toolkit 1.1:
/* Send ping command. The reply will be ping_command_reply function. */
silc_client_command_send(client, conn, SILC_COMMAND_PING,
ping_command_reply, context,
1, 1, server_id, server_id_len);
Notify arguments
In Toolkit 1.1 the following notify arguments have had changes. See
the Notify Arguments for details. You should
go through your application and change the handling of the following notify
messages for Toolkit 1.1.
SILC_NOTIFY_TYPE_NICK_CHANGE
SILC_NOTIFY_TYPE_CMODE_CHANGE
SILC_NOTIFY_TYPE_SIGNOFF
SILC_NOTIFY_TYPE_SERVER_SIGNOFF
The basic changes in notify arguments from Toolkit 1.0 is that the
Toolkit 1.1 parses various lists and other raw data for the application as
opposed to sending them in the raw format. This makes programming easier.
Command reply arguments
In Toolkit 1.1 the following command reply arguments have had changes. See
the Command Reply Arguments for
details. You should go through your application and change the handling
of the following command replies for Toolkit 1.1.
SILC_COMMAND_WHOIS
SILC_COMMAND_INVITE
SILC_COMMAND_STATS
SILC_COMMAND_JOIN
SILC_COMMAND_CMODE
SILC_COMMAND_BAN
SILC_COMMAND_DETACH
SILC_COMMAND_USERS
The basic changes in command reply arguments from Toolkit 1.0 is that the
Toolkit 1.1 parses various lists and other raw data for the application as
opposed to sending them in the raw format.. This makes programming easier.
Other changes in client library
There are many other smaller changes in Toolkit 1.1 that require you to
change your application when porting from Toolkit 1.0. We are not listing
all of them here but briefly mention some API changes.
Listing channel private keys now return SilcDList instead of an array.
The key agreement API has changed a little bit and is now more cleaner and
supports all the features that are needed in full featured key agreement.
The silc_client_peform_key_agreement_fd has been replaced by
silc_client_perform_key_agreement_stream.
The private message key API has slight changes also. It is no longer
necessary for the caller to specify whether the private message key is for
responder or initiator use.
The file transfer API has changed a little bit and is now more cleaner and
supports all the features that are needed in full featured file transfer.
It is now easier to send files when you are behind NAT when you can
specifically define the IPs that are used in both file sending and
receiving.
As a new function silc_client_nickname_format can now be used to format
the nickname of a client entry. The client library automatically formats
the nicknames but in some cases application might like to change the
nickname of a certain client entry.
Utility library, lib/silcutil/
The Utility library (runtime library) has had several changes and has several
new interfaces. Some interfaces has also been removed or moved to some
other library. Removed interfaces rae: silcprotocol.h and silcsockconn.h.
Moved interfaces are: silcapputil.h and
silcvcard.h.
SILC Async Operation Interface
A new asynchronous operation API (silcasync.h)
has been added. It can be used to control asynchronous operations, like
to cancel them. Many asynchronous routines in SILC Toolkit now return
SilcAsyncOperation context so that the operation can be controlled by the
caller. It especiallly provides a generic way to cancel asynchronous
operations which can be difficult.
SILC Atomic Operations Interface
A new atomic operations API (silcatomic.h)
has been added. It provides routines to perform various operations on
integeres and pointers atomically.
SILC Data Stack Interface
A new data stack (memory pool system) API
(silcstack.h) has been added.
It provides a fast memory allocation system. Many routines in the SILC Toolkit
are SilcStack aware thus enabling them to use the SilcStack as their source
for memory allocation. All routines that are SilcStack aware automatically
revert back to normal memory allocation if SilcStack is not given as
argument. See silcstack.h for list of utility routines that support SilcStack
by default.
SILC Condition Variable Interface
A new condition variable API (silccond.h)
has been added. It provides condition variables for multithreaded
applications.
SILC Stream Interface
A new abstract stream API (silcstream.h)
has been added. The SilcStream
provides an abstract way of representing different kinds of streams. The
API provides functions that can be used to read, write, control and destroy
streams. The API is not used to create streams but separate interfaces
exist for streams that use the SilcStream abstraction. For example,
socket stream and file descriptor stream exist.
SILC FD Stream Interface
A new file descriptor stream API
(silcfdstream.h) has been added. It
provides a blocking and non-blocking file descriptor stream through the
SilcStream abstraction.
SILC Socket Stream Interface
A new socket stream API
(silcsocketstream.h) has been added.
It provides a blocking and non-blocking socket stream through the SilcStream
abstraction.
SILC FSM Interface
A new Finite State Machine API
(silcfsm.h) has been added. It provides
an FSM that can be used to implement all kinds of machines and protocols.
The machine also supports threads, and threads that are actually executed
in real system threads. The SILC FSM API also supports asynchronous
events.
SILC Time Interface
A new SILC Time API
(silctime.h) has been added. It provides utility
functions to retrieve and represent time in different ways. It supports
Universal and Generalized time string creation and parsing and adds a new
SilcTime structure to represent time.
SILC Snprintf Interface
A new snprintf API
(silcsnprintf.h) has been added. It
provides snprintf and other string formatting routines.
SILC Mutex Interface changes
The SILC Mutex API (silcmutex.h)
has several changes. A support for read/write locks has been added
(SilcRwLock). Also silc_mutex_assert_locked function is added.
SILC Network Interface changes
The SILC Network API
(silcnet.h) has several changes. The API is almost
entirely rewritten and most of the old functions have been removed. The
API has now both TCP and UDP support, and as previously supports IPv4
and IPv6. New functions are silc_net_tcp_create_listener,
silc_net_listener_get_port, silc_net_close_listener, silc_net_tcp_connnect,
silc_net_udp_connect, silc_net_udp_receive, silc_net_udp_send.
SILC Scheduler Interface changes
The SILC Schedule API
(silcschedule.h) has several changes. The
scheduler has been entirely rewritten but most of the API remains the same.
The SILC_TASK_GENERIC and SILC_TASK_CALLBACK_GLOCAL have been removed.
The way signal are dispatched has been changed. The SILC Schedule is now
able to itself dispatch all signals. New functions are
silc_schedule_task_add_fd, silc_schedule_task_add_timeout,
silc_schedule_task_add_signal, silc_task_del_by_all,
silc_schedule_get_fd_events. The functions silc_schedule_signal_register,
silc_schedule_signal_unregister and silc_schedule_signal_call have been
removed.
SILC Types Interface changes
The SILC Type API (silctypes.h) has several
changes. The bool type is replaced with SilcBool. Sockets are now
represented by SilcSocket.
SILC String util Interface changes
The SILC string utility API
(silcstrutil.h) has changes. The PEM encoding
and decoding routines has been renamed, silc_base64_encode,
silc_base64_encode_file and silc_base64_decode. The silc_mime_parse has
been removed. A new silc_string_split function has been added.
SILC Utility Interface changes
SILC File Util Interface changes
The SILC file utility API (silcfileutil.h)
has changes. A new function silc_file_set_nonblock has been added.
SILC List and Dynamic List Interface changes
The SILC List (silclist.h) and SILC Dynamic List
(silcdlist.h) APIs have changes. New functions silc_list_insert and
silc_dlist_insert have been added.
SILC Buffer Interface changes
The SILC Buffer API (silcbuffer.h) has several
changes. The SilcBuffer
structure no longer contain the buffer length and true length fields
but silc_buffer_len() and silc_buffr_truelen() macros are available
instead. Also silc_buffer_data(), silc_buffer_datalen(), silc_buffer_purge(),
silc_buffer_reset(), silc_buffer_start(), silc_buffer_end() and
silc_buffer_enlarge() has been added. The API also supports SilcStack.
SILC Buffer Formatting Interface changes
The SILC Buffer Formatting API
(silcbuffmt.h) has several changes. The
silc_buffer_format now automatically allocates memory to the destination
buffer if it does not have space. Also new the following new formatters
have been added: SILC_STR_DATA (replaces SILC_STR_UI_XNSTRING),
SILC_STR_BUFFER, SILC_STR_FUNC, SILC_STF_OFFSET and SILC_STR_ADVANCE.
The API also supports SilcStack.
SILC Memory Interface changes
The memory allocation API (silcmemory.h) has
several changes. It supports now SilcStack as memory source. Also all
memory allocation routines can now fail and return NULL as opposed to fatally
failing when memory allocation fails.
SILC Application Utility library, lib/silcapputil/
A new SILC Application Utility library has been added. It provides
various application specific utility libraries that are not part of
the runtime library (lib/silcutil/). The interfaces in the Application
utility library were in other libraries in Toolkit 1.0 and the library
does not contain any entirely new interfaces.
SILC Application Utility Interface
The silcapputil.h contains various application
utility functions. It
existed in Toolkit 1.0 but some of the APIs has been changed. The
silc_create_key_pair, silc_load_key_pair and silc_show_public_key APIs
has changed. A new silc_show_public_key_file has been added. Functions
silc_identifier_check, silc_identifier_verify, silc_channel_name_check,
silc_channel_name_verify, silc_get_mode_list silc_get_status_message,
silc_get_packet_name, silc_get_command_name, silc_parse_version_string,
silc_version_to_num, silc_client_chmode, silc_client_chumode,
silc_client_chumode_char and silc_id_render has been moved from other
libraries into this interface in Toolkit 1.1.
SILC ID Cache Interface
The ID Cache interface (silcidcache.h) has
been moved from lib/silccore into lib/silcapputil/.
SILC Key Repository library, lib/silcskr/
A new SILC Key Repository library has been added. The library provides
a SILC Key Repository API (silcskr.h) which provides
a repository for storing and retrieving public keys.
SILC VCard library, lib/silcvcard/
A new SILC VCard library has been added. The SILC VCard API has been
moved from utility library to own library in lib/silcvcard/.
SILC HTTP library, lib/silchttp/
A new SILC HTTP library has been added. The library includes SILC HTTP
Server Interface and SILC HTTP PHP Translator Interface.
SILC HTTP Server Interface
The SILC HTTP Server API (silchttpservder.h)
provides a simple HTTP server implementation for applications that want to
integrate a small HTTP server.
SILC HTTP PHP Translator Interface
The SILC HTTP PHP Translator API (silchttpphp.h)
provides PHP translates PHP code into HTML. It can be used to serve PHP
pages in HTTP server.
SILC ASN.1 library, lib/silcasn1/
A new Abstract Syntax Notation One (ASN.1) library has been added. The
library provides SILC ASN.1 encoder and decoder interface and SILC BER
encoder and decoder interface.
SILC ASN.1 Interface
The SILC ASN.1 API (silcasn1.h) provides ASN.1
encoder and decoder. The interface provides efficient encoder and decoder
and is support SilcStack as memory source. The interface is simple and it
supports almost all ASN.1 features.
SILC BER Interface
The SILC BER API (silcber.h) provides BER/DER
encoder and decoder. It is integral part of the ASN.1 library and the ASN.1
encoder and decoder.
|
SILC Toolkit Reference Manual
Introduction to SILC RNG