Development

Alpha 3 Update

The SAFE Network is a self-configuring and autonomous network designed to manage all our data and communications without any human intervention and without intermediaries. In the past, it has been likened to a cyber brain, complex in nature and requires all component parts to all work in unison to achieve our vision of digital privacy, security and freedom for all.

No need to reinvent the wheel
It has never been our intention to invent everything ourselves, although this has quite often ended up being the case. SAFE Network features like Self Authentication, Self Encryption and Disjoint Sections are testament to that fact. However, we have always worked to reuse or repurpose existing technology if it worked well within the network design. No need to reinvent the wheel if you don’t have to!

In fact, one of the SAFE Network’s fundamental structures is Kademlia, a Distributed Hash Table (DHT) designed by Petar Maymounkov and David Mazières in 2002 that ‘specifies the structure of the network and the exchange of information through node lookups’. Our network design has required us to upgrade the original Kademlia design and introduce Disjoint Sections as mentioned above, but being able to take a design standard and repurpose it saves considerable resource and time.

In order for the network to reach an agreed state and to achieve consensus, members (nodes) of the network vote and democratically agree on a network state. These votes take the form of messages and may determine who has authority to access a file or to store a piece of data for example. Disjoint Sections requires an algorithm that asynchronously accumulates and orders the messages it receives from its members. This has been a challenging issue for our engineers to overcome and has in part required MaidSafe to release so many iterations and test networks (around 25) to this point.

Ordering algorithms
We have spent a lot of time working in this area and have looked at many state of the art ordering algorithms in addition to working on our own. We have found a few that are very promising and could be adapted and one that we feel will work well within the SAFE Network. We believe this to be significant in that it would radically simplify our code, save a lot of development time and minimise the number of future testnets. This would enable us to reach our future milestones (the next of which is Alpha 3) much more rapidly and with much greater confidence as we can focus on the network’s unique features rather than dealing with the complexity of our existing approach which requires us to handle order related issues individually.

Unfortunately the solution we have found conflicts with our own open source ethos and is in fact closed source. We are currently working on a solution to this issue and while this effort is ongoing we will respect the developers license and keep the consensus ordering mechanism closed source and in a private repository as a temporary measure to enable the rest of the team to complete the Alpha 3 features.

We will never launch the Network with code that is closed source – so please be assured this is only a staging point on the road to achieving that result. This is an area that has caused us much internal debate but ultimately we have made the decision that we feel gets us to a full release more quickly. In essence, we believe the prospect of the end result justifies the route – and as the pressure builds across society for a solution to the status quo (as evidenced by the current Facebook/Cambridge Analytica story), our role remains the same as it has ever been – to create a system that is free to everyone to use around the globe in which each individual has the opportunity to gain full control of his or her digital rights.

Announcing SAFE DevCon 2018

 

SAFE DevCon 2018

We’re happy to announce that we’re running our second DevCon in Ayr!

On Monday 23rd April 2018, the global MaidSafe team will be descending on Ayr Racecourse for an event that’s focused primarily on developers who are currently working (or wanting to work) on SAFE Network apps.

Whilst we’ve held various events previously (not least the conference in Asia in 2017) and the community has been running meetups around the world for the past few years, it’s the first time that we’ll have all the team in one location for a day solely dedicated to the SAFE Network.

The full agenda will be published in a couple of weeks – but you can assume that we’ll have a wide range of speakers from across the different areas of the protocol running through the existing state of the SAFE Network, laying out the plans for the year ahead and highlighting a few of the many apps that are flourishing independently from developers around the world.

For 2018, we’ve decided to keep the numbers small. For developers that can make their way to Ayr, we’ll be covering the costs of two night’s accommodation around the Conference. If we have too many applicants, we’ll choose by drawing names from a hat and get back to each applicant (hopefully by Thursday 22nd February) to confirm whether they’ve been successful in securing a ticket.

And those of you within the Community who live around the world and won’t be able to travel, don’t worry – we plan to stream the whole event live and share the videos afterwards.

So if you’d like to come along, take the unique opportunity to meet the full MaidSafe team and hear the very latest about all things SAFE Network, please email us directly at outreach@maidsafe.net (including your safenetforum handle).

 

SAFE Network Autumn/Winter 2017 Update

It has been a busy few months with the SAFE Network, we have had updates on the Network, the APIs and Browser as well as internal changes here at MaidSafe.  

As we continue to grow we have taken the decision to open an office in India.  As we already have a number of key front-end developers in Chennai this new office space will allow us to further develop this team and enabling greater internal collaboration. Being based in  the technological hub of Chennai will help us attract some of the best talent as well as interact with other tech firms, industry events and meet-ups.  We have found a great space in an IT business park which can house up to 30 staff, primarily the front end team.  Renovations and design of these premises is set to be completed by late January 2018. This is a very exciting project and we will be sharing pictures of the office, the renovations and updates on the forum.

As many of you who follow us on the forum and social media will know we have been experiencing some big changes in Scotland too.  We recently moved from our previous office in Troon to a new place in Ayr.  Although this is not a big move geographically it is a bigger and more comfortable space which we hope will allow us to continue to grow both the operations and development teams.  In October we brought onboard a Digital Marketer to help us improve our market presence and marketing strategy.  Alongside Nick, Sarah has been developing the near term marketing strategy; there is a focus on increasing our brand awareness, educating people about the network, improving the understanding  of Safecoin and its unique features, as well as showcasing the world’s first and only autonomous data network to a wider audience.

In line with this Nick has been busy promoting the network from Glasgow to Google. We attended and spoke at the  Scottish Blockchain Meet up in early November in Glasgow.  There was a strong turnout and some very interesting questions and discussion.  If you would like to join the next meet up, keep an eye on our Facebook. On 30th November Nick headed to Dublin to give Google an introduction to the SAFE Network.  Employees from across the globe joined the meeting through a video conference and many more watched the video subsequently.

To maintain the momentum, the marketing department will be further strengthened by Dug Campbell later this month. Dug has worked for a number of leading technology companies including Sky Scanner and MiiCard and has been an advocate of Bitcoin for a number of years, organising Scotland’s first Bitcoin conference in 2014 and speaking about it later that year at TedX Glasgow. With a wealth of knowledge and experience Dug promises to be a great addition to the team.

We are also now recruiting for a number of other roles in our Ayr HQ.  We continue to  look for a UI/UX Designer, Software Support Engineer and a Testing and Release Manager. To start a conversation and join this exciting journey email outreach@maidsafe.net

There has been extensive updates across the network recently and the team are continuing to make strong and consistent advances.  Since the release of Alpha 2 in mid-September there has been a number of test networks and key updates and changes.

Following the release of Alpha 2 there has been two new SAFE Browser releases, V0.7.0 and V0.8.0 both of which made incremental updates and fixed bugs. Following extensive discussion across the dev teams and community it has been decided that a custom Browser should be created following the reorganisation of Beaker since our fork that has made maintaining our current browser difficult.  While we did consider other browsers including Firefox and Brave the work-around were too extensive.  We are therefore creating a Electron-based browser and a Proof of Concept is currently in external testing. This pre-release can be downloaded here.

There has also been updates to the SAFE Apps Nodes.js and SAFE Web APIs following changes to the SAFE Client Libraries. SAFE App Nodes.js has been updated to take advantage of the master branch of the safe client libraries. Further to our commitment to open source and community focused development Nodes.js allows everyone to develop self-contained web apps in Javascript. The DOM API has been updated to reflect changes made in app nodes, this will reduce the number of handles the DOMS API need to expose and will simplify the web apps code. We hope that these changes will improve the developer experience.

The next big challenge for the team is the creation of data chains and we are now nearing the final stage. As you may remember from our previous blog, data chains allow the network to republish data if it should ever lose power.  We have begun coding the features to better express the fundamental and continue to test the design. These designs will be incorporated into the alpha 3 release.  A deeper dive on the data chains design is available here.

In Crust the p2p library integration is almost complete.  This will move reliance away from the slow and unfriendly peer to peer world of TCP and provide more reliable NAT traversal. Integration is a significant step towards a secure multi-protocol, randomised port encrypted network library which will be a great help to many projects, and offer greater security and privacy for users..

We once again want to thanks the SAFE community for your continued support as we work to create the SAFE Network and the levels of data privacy and security you all deserve.

The MaidSafe team.

 

SAFE Network Alpha 2 (The Authenticator) Launch Announcement

Today we are excited to be releasing the next major milestone in the roll out of the SAFE Network, Alpha 2 – The Authenticator.

This latest step is a culmination of a significant amount of hard work from the MaidSafe team, and much testing from the superb SAFE Network community. The result is a new network access control mechanism, the Authenticator, which enables users to securely authenticate themselves onto the SAFE Network, while protecting their network login credentials from apps.

Bundled with the SAFE Browser, the Authenticator supports Windows, OSX and Linux, and as many will have seen from last week’s update, now also supports Android, with iOS support in progress. As we mentioned last week, the intention with the mobile example apps is to confirm mobile platform support, we will provide the docs, tutorials and APIs that mobile app developers would expect in due course.

The desktop version of the SAFE Browser will come with two tutorial applications. The Web Hosting Manager and SAFE Mail. These apps will be familiar to those who have taken part in recent test networks. The Web Hosting Manager allows users to create their own public ID, and upload and publish content instantly. SAFE Mail provides end to end encrypted email using the public key of the recipient to encrypt the message.

The two Android applications provided with this release are the Authenticator, and SAFE Messages, an example application that demonstrates end to end encrypted mobile email.

As has been the case with more recent test networks you will be required to obtain an invite code in order to participate in this Alpha. In order to do this you will need to have a basic user account on the SAFE Network forum. For those new to the SAFE Network and the forum, the following link tells you how to get this. This measure is in place to prevent the network being flooded with data prior to the network being fully featured.  

As you may have noticed, the installers for today’s release are on our new web page (we hope you like it) and tonights forum post contains a full breakdown of information relating to this release. We hope you enjoy using Alpha 2 as much as we enjoyed creating it!

SAFE Network: Mobile Tech Preview

As MaidSafe continues its progression in the role out of the network, we have hit another important milestone that we would like to share. We now have SAFE mobile applications, running on Android and iOS, and today we have released some demonstration apps to showcase this progress. iOS requires some code updates and app certification to be ready for user testing and is currently limited to testing via the iOS simulator.

It is important to note that these apps should be considered as a technology preview, a very useful proof point for us that the SAFE Network accommodates mobile devices. This is the culmination of several changes that have been made over the past 9 months, including a new data type and a new access mechanism, in the form of the ‘Authenticator’. In time we will provide mobile developers with the tools and documentation that they would need and want to start developing SAFE mobile apps. In the meantime, please refer to today’s dev update for instructions and requirements for running these apps for yourself.

The Authenticator
The first of these applications is the Authenticator. This is the focus of the imminent alpha 2 release and the mechanism by which users securely access the network, while maintaining control of each SAFE applications access to their data.

""

SAFE Messages
The second application is a stripped back and simple mail app. It provides end to end encrypted messaging that uses the public key of the recipient to encrypt the message, ensuring that only the recipient can read its contents.

Alpha 2
The mobile tech preview comes at an exciting time in MaidSafe’s development roadmap, a welcome lead into alpha 2 which we will be releasing next week, on Thursday the 21st of September. This latest alpha will incorporate the Authenticator, a new SAFE Network access mechanism that is network enforced, and as you can see from today’s announcement mobile friendly. We look forward to providing more detail next week.

At MaidSafe, our development approach has been different to many other projects in the space. We have focussed on the hard problems first. This is not a criticism, just recognition of a different approach. Rather than putting out a network that gives little thought to the security of the data on it, or ignores the issue of how it will scale to millions of users, we have prioritised finding solutions to these big questions up front. This may create the appearance that we are moving slower than many of the other larger infrastructure projects in this sector, but in tackling the more challenging issues from the outset, in a methodical and transparent way, we anticipate being well placed to provide the decentralised infrastructure of the future.

SAFE NETWORK DEVELOPMENT SUMMARY – AUGUST 2017

Since the last blog update in May we have published new test networks that are helping us to evaluate much of our recent development work. If you recall, we made several changes to be able to accommodate mobile devices as network clients. These changes included the addition of the Authenticator (a secure access mechanism that is bundled with the SAFE browser) and a new network data type – mutable data – as well as a significant number of changes within the APIs.

Test 17
The current network, test 17, was introduced initially to a small number of forum users, but has since been scaled out in order to accommodate more users. Updated mid July (13th) and re released based on initial feedback (and barring a few minor bugs), test 17 has behaved as anticipated and we’re very encouraged by its stability. We intend to keep a test network in place from now on to enable app developers to develop against this network, rather than resorting to running apps locally.

Forum member Zoki has put together a couple of videos which he has posted on YouTube that demonstrate the use of the Authenticator and the Web Hosting Manager, as well as viewing a few SAFE websites along the way. The Authenticator enables users to create their own network credentials without the involvement of third parties and provides access to the test network.

DNS, but not as we know it
The Web Hosting Manager facilitates users creating their own public ID and service that they can then upload content to and publish for other network users to view. This feature demonstrates a differing approach to the Domain Name Service (DNS) used on the existing Internet that is managed by several DNS providers, such as Dyn and Verisign. Within the SAFE Network, this Decentralised Naming Service, enables web site owners to create their own domain without the involvement and cost of third parties and enables instant publishing of data.

If you are a SAFE Network forum member of trust level 1 and higher, you will be able to participate in this test and play about with these demo apps for yourself, and the following thread contains links to many of the websites published by other forum members.

SAFE email client
The second video produced by Zoki demonstrates the Email application, which is an end to end encrypted messaging app that uses the public key of the recipient to encrypt the message, ensuring that only the recipient can read its contents. Currently using nodes managed by MaidSafe in test 17, SAFE email in future alphas will be decentralised, ensuring that no central entity can view or control access to your communications.

It is important to note that these example applications are intended as tutorials which demonstrate the features of the network while guiding application developers to create more fully featured and polished apps with the SAFE Browser DOM APIs.

Data Chains
What we currently have in test 17 is likely to not have too many more changes before we move to alpha 2. As mentioned above, we are very encouraged with the stability of this network. In tandem with much of the work above the team has been working on a feature called Data Chains. You may remember from our previous blog post that this is a feature we anticipate will ultimately enable the secure republishing of data should the network ever lose power, as well as providing validation that data has been stored on the network. The team has considered multiple implementation options, and subject to simulation tests, has agreed an approach and have started the implementation. Testing of this new Routing design is likely to be incorporated within alpha 3. For plans beyond this, please refer to our roadmap.

Recruitment
For those who regularly go on our forums you will notice an increasing number of new team members. Recruitment continues to receive significant focus as we scale the team to increase the speed and quality of the network roll out while also spreading the load more evenly across the team. As such, we have brought on board some operations staff at our HQ in Scotland and continue to grow the team overseas, who are currently based anywhere from Australia to Argentina!

We now have 23 people working with the company, but we are still looking for Network Engineers. If you are proficient in Rust, or have experience with C or C++ and have experience within P2P architectures, please visit our careers page for more details on how to apply.

Well, that concludes this update, we really appreciate the continued support of everyone in the SAFE community (investors, testers, forum members). As you know we are doing everything possible to expedite the network rollout and giving you the privacy, security and freedom you all so richly deserve.

SAFE Network Development Summary – May 2017

We’ve had quite few requests on social media and on email these past few days requesting updates on development progress. These messages serve to remind us that not everyone has the time or the inclination to read the weekly development updates which we post each Thursday onto the forum. So many projects, so little time! So the intention with this post is to provide a summary of the most recent events and our hopes and expectations moving forward.

Image: Richard Tilney Bassett

Roadmap
The best place to start is our development roadmap, which we updated and published late last week. This web page tries to encapsulate all the complexities of development over time on 1 page so it’s pretty high level, but it is this snapshot view that most people seem to appreciate. You will notice that the roadmap outlines the major aspects of development and a rough indication of the order in which we anticipate tackling them.

You will also notice that we haven’t included timescales. In the past we have provided timescales for ‘launch’ of the network. These have always been wrong despite our best efforts. We have found it difficult to estimate timescales since, we believe, so much of what we have been working on is brand new technology, sometimes completely bespoke, and other times building on the work of other projects. Testing is also interesting, it really helps us understand more about how the network fits together and how it is utilised by our community, but invariably leads to more tweaking and testing with previously unplanned and unknown rework and test durations.

We believe that publishing release dates that have a high degree of uncertainty attached is not helpful to anyone and can cause more frustration than not publishing them at all. Network related development is typically where the biggest black holes are and as we get into incremental development client-side, we anticipate time scales will become more predictable.

Stable decentralised network
In late March we released test 15, a network that incorporated both data centre resource as well as enabling user run vaults. Within this release, users were also able to run the SAFE Browser, Launcher and demo app, which continue to facilitate the storage of private and public data, as well as create public ID’s and publish SAFE websites.

After 3 days of running a stable network without any lost data we realised we had reached an important milestone. While we had done this extensively in private tests, it was fantastic to see it running publicly and see the community reaction to it. Of course, life has a sense of humour and shortly after it became apparent that a script had been written that created fake accounts and filled the relatively small network with data, stopping the creation of new accounts or the uploading of new data. This was really helpful to us as it enabled us to find out what happens to the network when it reaches capacity in a real world setting. The fact that it behaved as expected was reassuring, although we’d be lying if didn’t admit to finding the spam attack a little frustrating. This is of course something that the integration of safecoin would stop, as the requirement to ‘pay’ to store data will make the attack expensive, while the incentive of safecoin to farmers would lead to a significantly bigger network.

What now?
Looking forward we are currently focussed in 3 main areas:

  • Catering for mobile devices.
  • Enabling greater user participation.
  • Improving the resilience and robustness of the network.

Mobile
The patience app developers have shown to this point is soon to be rewarded. The process of converting our APIs away from a REST paradigm to SDKs was essential to cater for mobile devices, as the requirement for REST APIs to maintain state would not have worked with mobile devices that disconnect and reconnect regularly. Users of the SAFE Network will gain access through the Authenticator, a secure gateway that protects user credentials from the application itself. The Authenticator is currently being bundled with the SAFE browser and will enable users to securely authenticate themselves onto the network, or enable them to browse publicly available data without logging in.

To implement Authenticator the team required to add a new data type, mutable data. The new data type improves the network efficiency, saves bandwidth, and provides the granular access control required by mobile platforms.

With mobile devices being so ubiquitous throughout the world, enabling mobile client access to the network, mutable data has been receiving significant focus. From a resource provision perspective, both alpha and beta versions of the network will require laptop and desktop and in time single board computers to earn safecoin when it is released. In time, we will look at enabling mobile devices being able to farm for safecoins when plugged into a power outlet and when in range of WiFi, however, as we will detail below this is not a priority for now.

More alphas
Some of the example applications that have been created are currently being ported to suit the new data type and to be compatible with the new APIs. The team are updating the documentation and are testing the applications using a mock network, and they seem to be far more stable than previous iterations which looks positive. We anticipate alpha 2 will encompass the new Mutable Data type and Authenticator, SAFE Browser DOM APIs and Node.js SDK, along with example apps, tutorials and documentation.

Image: Clint Adair

Alpha 3 will see our focus shift onto enabling a greater number of users to run Vaults from home by integrating uTP. Presently users must TCP port forward, or enable UPnP on their routers which requires a little set up in some cases. Adding uTP support will make for a more seamless process for many while making the network accessible to more users. uTP is used in some BitTorrent protocols and when implemented effectively helps to mitigate poor latency and facilitate the reliable and ordered delivery of data packets.

During this phase we will also integrate node ageing, a feature that make the network more resilient to consensus group attacks. The team will also implement the first part of data chains, a feature that has been planned for a while which it is anticipated will ultimately enable the secure republish of data should the network ever lose power, and to provide validation that data has been stored on the network.

Looking ahead
Beyond alpha 3 we will focus on:

  • Data Chains, part 2.
  • Data republish and network restarts.
  • A security audit of the network
  • Test safecoin
  • Real-time network upgrades
  • Network validated upgrades

As has been the case to this point we will continue to release multiple test nets regularly between each alpha network to prove the technology in a public setting, and to mitigate against the code regressing.

We continue to be grateful to the huge support of the people that take the time to run these networks and report back, you all know who you are!