A High Performance Data Distribution and Sharing Solution with Sector
June 14, 2010 6 Comments
Over the years, many users have used UDT to power their high speed data transfer applications or tools. Today, with Sector, we can provide an advanced data distribution and sharing application, in addition to the UDT library. This solution currently works on Linux only, but we will port it to Windows in the near future, first the client side, then the server side.
Here are the several simple steps you can follow to set up a free, open source, advanced, high performance, and simple-to-use data distribution and sharing platform:
2. Set up a security server, which allows you control the data access permission, including user accounts, passwords, IP access control list, etc. You can also set up an anonymous account for your public data.
3. Set up one or several Sector master servers, which can be on the same computer that hosts the security server and the data (slave server).
4. Set up Sector Slave servers on the computers that host your data. Unlike FTP or most commercial applications that supports only a single server, Sector allows you to install the servers on multiple computers, even 1000s of them, and yet provides a uniform namespace for the complete system.
5. Install client software on your users’ computers and mount the Sector file system as a local file directory using the Sector-FUSE module. Your users can browse and access the data in Sector just as browsing and accessing data on a local directory by using file system commands “ls”, “cp”, etc. They will not feel the existence of Sector, although this “local directory” can actually run on 1000 servers across multiple continents!
All the data transfer occurring between the clients and the slave servers are on top of UDT. Therefore, data transfer throughput can be guaranteed even over wide area networks.
If you have any questions, please post them on the Sector project forum.