January 31, 2010 Leave a comment
Sector version 2.0 is in the QA stage and will be released soon. This is a major milestone for the Sector project: 1) Sector 2.x is ready for production; 2) the code structure is re-designed to accept contributions from a large community.
Since the last version 1.24a, we have added several new features, including on-disk metadata and in-memory objects. Previously, Sector keeps metadata in memory, which is very fast but may limit the number of files supported in the system. The new on-disk metadata will support much large number of files. Both metadata systems will exist in version 2.0 with the default set to the in-memory one for performance reason (we expect that in the majority of systems the in-memory metadata structure will be enough to hold all the file information).
In fact, in Sector 2.0, it is easy to support a new metadata structure by inheriting from a base C++ class. Switching between different metadata management can be done via the configuration files. In addition, because all the metadata system define the same interface, it is possible to have different metadata structures on different nodes.
Version 2.0 introduces in-memory objects. A UDF can create an in-memory object (a pointer to an allocated data structure in memory) while another UDF can access it. This can significantly increase certain iterative algorithms. The in-memory object can be released by a UDF when it is not needed any longer.
Code Structure Reorganization
We have reorganize the code structure so that the Sector system can be developed by a large development group. Most functionalities, including security control, metadata, master synchronization, services, and client API have been separated from each other . These parts uses internal protocols to communicate between each other. By this way, each part can be independently developed or improved as long as it implements the protocol interface.
In version 2.0, only a single header file is needed to include when programming Sector. This makes it easy for a standard installation (one header file and one library file in the system directory).
During the reorganization, we have also improved the code quality. The new clearer and more loosely-coupled structure make Sector less prone to design and programming bugs.
Plans for 2010
We will continue to improve the software by providing better performance, more reliable software, and more detailed documentations. Minor versions (2.1, 2.2, etc.) may be released once a quarter.