These design principles used by Amazon to meet Amazon S3 requirements are pure gold: simple yet powerful.
• Decentralization. Use fully decentralized techniques to remove scaling bottlenecks and single points of failure.
• Asynchrony. The system makes progress under all circumstances.
• Autonomy. The system is designed such that individual components can make decisions based on local information.
• Local responsibility. Each individual component is responsible for achieving its consistency; this is never the burden of its peers.
• Controlled concurrency. Operations are designed such that no or limited concurrency control is required.
• Failure tolerant. The system considers the failure of components to be a normal mode of operation and continues operation with no or minimal interruption.
• Controlled parallelism. Abstractions used in the system are of such granularity that parallelism can be used to improve performance and robustness of recovery or the introduction of new nodes.
• Decompose into small, well-understood building blocks. Do not try to provide a single service that does everything for everyone, but instead build small components that can be used as building blocks for other services.
• Symmetry. Nodes in the system are identical in terms of functionality, and require no or minimal node-specific configuration to function.
• Simplicity. The system should be made as simple as possible, but no simpler.
Taken from: https://queue.acm.org/detail.cfm?id=3434573