The digital age has given us access to vast amounts of data, available through a multitude of online devices. We use dozens of different services on a daily basis, reaping the benefits of near-instantaneous data availability. However, making these services constantly and promptly available is becoming more challenging by the minute. Deployment of new apps and features requires ever-increasing storage space for the massive data they rely on. Correspondingly, the nature of usage and development of these digital services seems to be aggressively favoring cloud-based solutions. Thus, a new debate emerges: should you use Amazon S3 or local storage, and what to look out for? Here we cover the pros and cons of both options to help you make your storage choice, but first…

What is object storage?

Essentially, object storage is a type of computer data storage that allows you to store huge amounts of unstructured data. It differs from other storage architectures such as file systems and block storage, and its use is becoming more widespread. Stored objects almost universally include the data itself, followed by metadata and a globally unique identifier. This translates quite well into services requiring large amounts of images, videos, audio clips, or shared files. Think Facebook, Spotify, and Dropbox. Now, object storage is done locally or via cloud-like saving a photo on your phone versus on iCloud. However, note that all the corporations mentioned above need to serve millions of users simultaneously… and lightning-fast. On the other hand, there are businesses that don’t require such a huge scale, and they can opt for local storage. 

Let’s see what these two storage options have to offer, and which types of users would benefit from them.

Amazon S3 – anywhere, anytime

Amazon S3 (Simple Storage Service) is a cloud-based service whose main strengths are time-saving, security, and ease of use. Developers and IT teams like to use it because it’s intuitive, accessible, and allows for quick manipulation and deployment of objects. S3 comes with a rather elegant web service interface, which adds another layer of practicality in everyday use. The users simply select the appropriate location for storing their data. Immediately after, they create a storage bucket and deposit their objects inside, with the ability to access them anytime. 

Amazon S3 Pros

• Scalability – Often enough, file transfer servers are highly dependent on physical (hard) drives, which limits their space by default. For this reason, server admins tend to impose quotas or mandate regular archiving to save space. If a server maxes out on data, you need to add more hardware and this can result in downtime. Amazon offers virtually unlimited storage, and you won’t have to worry about guessing how much space you’ll need. For stuff you won’t likely use anytime soon, there’s the Amazon Glacier low-cost archive storage option.
• Availability – Local storage brings an ever-present fear that the hard disk you’re storing your files on could crash. Not only that, but your server could also crash and you won’t be able to access your files. Amazon S3 meets the availability threshold upwards of 99.9%, which virtually guarantees timely access to data.
• Backup options – Linking to the trait above, Amazon S3 offers easy backup options compared to local storage. It’s fully automated and allows you to access various older versions of your objects.
• Pay-as-you-store – Amazon’s service is designed in such a way that it doesn’t limit your storage. Instead, you simply pay based on how much storage you use at any given point in time. Regardless of your needs, you’d likely be much better off in terms of cash flow optimization. There would be no need to pay large upfront costs right away if you expect to require large storage consumption. Conversely, if you’re starting small, you can generate great cost savings early on and then increase your storage proportionately as you grow.
• Regulatory compliance – In case you have specific data protection and storage requirements (such as geographical limitations, for example), Amazon allows you to create regional buckets where you can store your data and be fully compliant within minutes.

Finally, Amazon S3 is quite secure. Keep in mind that this doesn’t mean you can’t make your local storage highly secure. However, it does mean that you gain a well-resourced partner in building your security system.

S3 Cons

Using Amazon S3 will result in additional latency compared to local storage. You’ll have to go through the internet every time to retrieve and use them. This latency can increase even more with geographical location distance. Even if your server is running on Amazon EC2, you will experience higher latency compared to physically attached local storage.

Some companies simply do not feel comfortable storing sensitive data outside their organization. Whatever the reasoning, the primary alternative is to use local storage.

Local storage – close to home

Storing your objects locally gives you two major advantages that counter the shortcomings of Amazon S3 mentioned above. You’ll get unparalleled low latency if you hook directly into your local storage infrastructure. Also, your privacy is yours to manage, so you can customize it to meet all your demands and requirements.

Keep in mind that this comes with quite a few costs, of which the greatest is time. You’ll have to deal with all of this directly and there’s no other option. Additionally, local storage does bring a few other downsides with it:

• Server processing load and size – In some cases, you can’t really anticipate the server load and storage size. Unfortunately, local storage will force you to “guess” this. If your guess is off, you risk running out of capacity at the worst possible time.
• Accessibility – If your local storage works, everything is great. However, if it fails or experiences downtime, the cause is likely going to be more serious. Resolving this takes precious time.
• Backup – Organizing backups is not a big deal in and of itself, but you’ll definitely require storage capacity for backup images. More costs, more work.

The verdict – Amazon S3 or local storage?

The Amazon S3 versus local storage debate will continue for the next couple of years. In the end, the needs of your organization will likely dictate which option you’ll choose. Undeniably, business processes and consumer trends are favoring the cloud, and Amazon S3 is the leading choice. So, if you’re just getting started, or you’re expecting to grow rapidly, Amazon S3 would still be your best bet.

If you are in any way involved in software development, you have likely heard the term “Shift-left” thrown around. Shift-left is a joint effort of developers and QAs (Quality Assurance) engineers to incorporate testing as early as possible in the software development life cycle. This approach fits in with the agile methodology allowing code development and testing in small increments. 

What is Shift-left?

More than a decade ago, the standard software development and QA cycle revolved around base levels. Developers wrote code for a “base level” and passed it to the QA team for testing. They would test the code and report any bugs and defects back to developers repeating this cycle as many times as necessary, often at great costs in time and resources. This approach made it very hard to give estimations, hit milestones and meet deadlines. 

Shift-left leverages the fact that bugs are easier and cheaper to fix the earlier they are detected. In order to find bugs early, you need to shift testing to an early stage of the project. When defects are detected early you can take corrective steps before things get too complicated. The easiest way to explain the idea of shift-left is an attempt to introduce testing as early as possible in the SDLC. Shift-left is also a mindset that testing should be a priority in every stage of development. 

Getting Started With Shift-left

As with any change in development approach and methodology, shifting left is an incremental process. It takes a lot of time, learning, organization, and participation of both tech and management teams. Depending on your current strategy, shifting left can require extensive changes to your team structure, roles, and organization. It can be a good idea to implement the shift-left approach on a small team before rolling it out in full. Regardless of scale, there are three basic steps that can get you going on the right path.

Step 1: Automate

In order to shift-left in a meaningful way, embracing automation is a must. Automation should not be limited to unit and functional tests, but also to testing the deployment of new builds. Automated builds with pre-loaded tests ensure lightning fast feedback about the quality and stability of new code. More importantly, it is a regression testing safeguard to make sure new changes did not break existing functionalities. 

Although automation might seem like a lengthy process when you have to start from scratch, it will speed up the SDLC and reduce time to market in the long run. Having automated tests in place takes some of the burden off the QA team and gives them more time for exploratory, usability, and other types of tests that are not easily automated. Automation reduces the number of bugs that sneak into production, and provide a higher level of confidence in product quality.

Step 2: Implement Coding Standards

To be successful, shift-left requires your entire team to be on the same page. The first step on this journey is to agree to a shared set of coding standards. Having clear coding standards not only makes code more readable but also makes code reviews faster- and time is money. Having an overarching coding standard also ensures new team members can get acquainted with code quickly. It also reduces potential merge conflicts when several programmers are working on the same code. 

There is no “one size fits all” coding standard that will work out of the box. Discuss with your team which tools would be the most relevant and effective for your codebase. There are a lot of static code analyzers on the market that highlight bad coding practices in real-time and offer fixes. If implemented correctly, this step will decrease the number of bugs and issues and decrease the amount of flaky and insecure code. 

Step 3: Test Early Reduce Cost

Teams should test code as early as possible to reap the benefits of shifting left. Given that approximately 85% of all bugs occur in the coding phase, testing at the end of the SDLC surrenders the chance to fix bugs when they most often occur.  Testing early requires a joint effort and a shift in mindset across the entire team. This approach aims to bridge the divide between developers and test engineers and promotes collaboration and knowledge sharing.

Continuous testing increases the confidence in the quality of the software, and gives managers and stakeholders a realistic picture of the state of the project. It also saves a lot of money. According to the 1-10-100 rule, prevention is cheaper than correction and correction is less costly than failure. This cost of failure formula states that investing 1$ in prevention saves $10 in correction and 100$ in failure 

Shift-left in Real Life

So, what does shift-left daily activities look like when a team fully integrates it into every part of the project?  A lot of waiting was once common for conducting tests for requirements, for new builds, and for completing development efforts before starting regression testing. This does not happen when test engineers are incorporated earlier in the development process.

When working as part of an integrated team, testers and developers share a similar cadence for more collaborative work. QA members take part in design and planning sessions with developers, even if they do not have in-depth programming knowledge. Getting familiar with the code and being part of the planning process puts testers in good stead to add quality to the project. Developers can help testers write unit and component tests and eliminate waiting around for deployments to a test environment.    

A direct result of separating testing from development is often a few common misconceptions. For example, measuring the performance of a QA team solely on the number of tests they run and bugs they find. Another one is assuming that a release is imminent when there is a decrease in the number of new documented tests. These misconceptions disappear when the whole team works on making testing happen earlier.   

You would be hard-pressed to find a modern website or application that does not use APIs. Short for Application Programming Interfaces, APIs define what type of requests can be made between online actors. They also determine how the calls are made, what conventions they follow, what formats need to be used, and much more. APIs are quite easy to reverse engineer. They are often well documented and accessible through public networks- making them enticing targets for hackers and other bad actors. In addition, APIs are extremely susceptible to denial of services (DDOS) attacks.

API security deals with privacy and access control as well as the detection of attacks and identification of vulnerabilities. As API infrastructure grows, the potential for gaps in security also increases. To avoid security breaches and ensure the safety of sensitive data, it is important to integrate API security into the development process. Here are some basic things you need to know before embarking on your API security journey.

Understand the Risks

APIs are designed to provide access to your data from an outside source, and as such are at risk of being hacked. Most APIs, unfortunately, have weak access control, and some are not protected at all. Many IT experts predict that API attacks will be the biggest cyber security concern in the near future. In some extreme cases, it is not only your data that is at risk; API breaches can also endanger your infrastructure. Bad actors can gain access to your network and escalate privileges to effectively become a superuser. With such power over your network, multi-level attacks can be leveraged to compromise your most sensitive data.

Whatever the severity- a data breach is a serious event that not only costs you revenue but can damage your brand and reputation. It seems that no one is immune to breaches. Many industry leaders have fallen victim to API attacks – Google, Instagram, T-Mobile, Uber, Verizon, Facebook, just to name a few.    

Common Web API Attacks

APIs are vulnerable to most of the same attacks that have become commonplace since the early days of the internet. The most common are:

Injection  

This is an attack where harmful commands or code is inserted (injected) into a program. It most commonly occurs in places where ordinary input is required- username and password fields are a good example. The most common injection attack is SQL injection, where an attacker breaks into an SQL database and gains control over it. Cross-site scripting, or XSS, is another type of injection attack that consists of exploiting vulnerabilities in order to insert malicious scripts into a web app or website. Injection attacks can be mitigated by limiting response data to avoid leaking sensitive data.

DDOS

DDoS, or distributed denial of service, is an attack that renders a website, network or system unavailable to users. This is done by flooding the system with an excessive amount of traffic that it was not designed to handle. APIs endpoints are prime targets for DDoS attacks. Rate and payload size limiting is a good defense against DDoS attacks. 

Man-in-the-middle

MitM attacks are based on intercepting traffic between two systems that communicate with each other. The aim of this attack is to circumvent mutual authentication by closely impersonating each endpoint. If this is done well enough to satisfy the expectations of both actors, the attacker acts as an invisible proxy between them. API MitM attacks can be carried out between the API and its endpoint, as well as between the API and web app (client).   

Credential Stuffing

Credential stuffing is a type of cyber-attack where stolen credentials (usually lists of usernames and passwords) are used to gain unauthorized access to accounts. This is most commonly done using large scale automated login requests. The same applies to API authentication endpoints.

Securing APIs – Best Practices

To maximize the security and safety of their APIs, organizations need to stick to well-established security controls and implement these basic security best practices.

Security First

All too often, API security is no more than an afterthought in the software development lifecycle. After code is committed, security is often looked at as someone else’s problem. This approach leads to vulnerable code, so it is important to prioritize security and build it into APIs as soon as you start developing them.

Keep your APIs Organized

If you do not put effort into keeping track of your APIs, vulnerabilities can go undetected. No matter if you have hundreds of publicly available APIs or just a handful, they need to be properly inventoried and managed in order to be secured. Frequent perimeter scans are recommended in order to discover and inventory APIs.

Strong Authentication and Authorization

Your API security is only as strong as your authentication and authorization solution. Public APIs are sitting ducks. Authentication breaches happen either when an authentication factor can be easily broken in to or when APIs do not enforce authentication. Security is often lax for private APIs. as they are intended for internal use only. APIs are entrance points to a company’s database, so access control is of paramount importance. OpenID Connect and OAuth2.0 are proven authentication and authorization tools that should be implemented if possible.

Least Privilege

The principle of least privilege dictates that users, programs, systems, processes, and devices should only be granted the minimal access needed to complete an action. This principle should be applied to APIs as well. 

Use TLS Encryption

TLS or transport layer encryption is a protocol whose main aim is to provide data integrity and privacy between applications. Some companies choose not to encrypt API payloads that they do not deem sensitive, but sensitive data such as login and credit card information should always be TLS encrypted.  

Remove Confidential Information

Information that you do not mean to be share should always be removed from APIs. When used as a developer tool, APIs often contain sensitive information like passwords, keys, and other credentials. All of this information needs to be removed before making the API publicly available. This step is sometimes overlooked, either because of time constraints or a lack of concentration. Scanning tools should regularly be used to minimize the risk of exposing information that needs to be kept secret.  

Don’t Expose Too Much Data

A common mistake made in API architecture is exposing more data than is necessary. This often occurs when data filtering is done via the user interface instead of by using endpoints. APIs should only return the minimal amount of data needed to carry out their function. 

Validate Input

All input data should be fully validated before passing from an API to an endpoint. Rate limiting is a great way to set a threshold above which requests will be rejected. 

We have already talked in detail about the nature, traits, and inner workings of the 5G network standard in a previous article. As this new technology spreads across the globe, people often wonder where it will prove to be the most relevant. Based on current industry trends, the greatest impact of 5G on business will be in services and automation. This infrastructure aims to facilitate the deployment of transformative services and integrate vast amounts of data with consumer-oriented products. Let’s see in which tech aspects we can expect the biggest opportunities and how they might manifest in practice.

The Need for Speed

A decade ago, people considered 3.5G completely sufficient to cover all consumer needs for mobile bandwidth. People accessed the internet via mobile devices in a relatively modest manner. Video streaming was just becoming popular and social media was yet to take off. The quality and latency were solid for that state of the market, but technology evolves. Take a look at just a handful of industries: gaming, video content production, and streaming, telemedicine. All of these rely on increasing bandwidth, low latency, interconnectivity, and ever-growing capacity. If you couple these trends with global events like the Covid19 pandemic, the need for fast and reliable information sharing becomes even more apparent.

Many are looking to work from home, but also increasingly from areas where they might not have wi-fi access. Imagine you want to work from a cabin in the woods, but you have no infrastructure apart from electricity. 5G would allow you to easily secure all the bandwidth you’d need for completing an important job function and for 4K video streaming. Big video conferences and resource sharing? Check. This opens a huge window of opportunity for augmented and virtual reality, where 5G will serve as a catalyst. These industries have to deliver seamless user experience, and increasing the attention to detail plays a key role. The richer the animations and the more lifelike the models, the higher the chance for market success. The same goes for large-scale event organization and the entertainment industry in general.

Ultra-low Latency

Besides the bandwidth boost, another advantage of 5G will be incredibly low latency. This plays a crucial role in developing new types of business applications and greatly improving existing ones. Low latency is paramount in any situation where we need rapid coordination between devices. This specifically impacts machines and devices which operate in physical space. For example, drones and other unmanned vehicles are able to communicate in rescue missions. Robotic surgeons would be able to conduct extremely precise and delicate procedures with a very low tolerance for errors.

Even more importantly, humans will be able to use these devices from any location with no fear of delays. Imagine a surgeon from Singapore using a remote laser to operate on someone in Oregon in real-time. Low latency also allows for the design and implementation of devices whose functioning needs to be instantaneous for maximum effect. Thus, hardware manufacturers are looking at new possibilities in various domains, but mostly focusing on sensors. Add to this a host of new control apps and you can expect a ton of new business startups opening in response to increased demand.

Higher Connectivity

Greater network capacity doesn’t just mean that we can serve more devices at the same time. It also translates into an ability to communicate between these devices and create even more valuable data. Using 5G, we could accommodate millions of sensors per square mile and enable even more IoT devices. This will allow better services in rerouting traffic, managing air pollution, and improving emergency responses. Companies with fleets or complex machinery could leverage this to speed up processes in their asset management activities. The impact of 5G will also extend to traditional jobs such as warehousing, stock management, and supply chain logistics. Increased connectivity further benefits small businesses as they will get access to cheaper and more effective service niches as 5G progresses.

Industries Looking to Benefit the Most

Based on current trends, 5G will eventually reach every single mobile user on the planet. The industries that will benefit most are:

Healthcare – 5G will facilitate remote surgeries from across the globe. Moreover, it will change the way people look at diagnostics. Patients will be able to upload complex patient data in real-time. This includes high-quality images, videos, and various other data types. On the other side, AI-powered algorithms will be able to process them in the cloud and provide rapid responses.

5G and Automotive

Self-driving cars used to be reserved for sci-fi novels and movies. Now, they’re being tested and perfected as we speak (Tesla, Waymo, etc.). This is one of the industries where close-to-zero latency plays a pivotal role. 5G will not only impact personal cars but freighters and heavy-duty trucking as well. Along with healthcare, autonomous vehicles are part of a huge industry where human safety is always a concern. 5G will also allow smart cars to monitor passenger health and simultaneously contribute to user wellbeing and resource efficiency.

5G and Retail

This industry is already undergoing tremendous changes as more and more people are buying online. However, there are still active limitations, almost universally related to items one would have to try on or customize. Retail could see a huge Augmented Reality (AR) boost which would help customers see how these products look in physical space. It would also allow for authentic virtual browsing and groundbreaking multi-sensory shopping experiences.

5G and Entertainment

Cloud gaming is already becoming a thing, and it is highly dependent on low latency. If latency is negligible, someone using 5G can easily enjoy games without the need for buying expensive hardware. The gaming-as-a-service sector is therefore poised to increase drastically. The expansion of socially driven AR/VR games is all but guaranteed in this sphere. Having tens of thousands of users being able to simultaneously communicate during events such as concerts will create new communities and bring a whole new layer of value.

Rest assured that this is just a glimpse of things to come. As with any new infrastructure, we expect the full impact of 5G on business to generate a host of new entrepreneurial ideas.