We need this built NOW!

One of the biggest worldwide stories of the last few weeks has been the spread of the deadly coronavirus, which has infected tens of thousands and killed hundreds. In the wake of the panic around this illness, governments around the world have gone to drastic measures to restrict its spread. In the Chinese city of Wuhan, which is the epicenter of this outbreak, officials have quickly scrambled to quarantine and treat those exposed to the coronavirus. The most significant accomplishment of this effort is that they were able to build an entire hospital in the span of just ten days.

Think about that for a second: a brand new, multi-story, 1,000-bed hospital was designed and built in just a week and a half. It took longer than that for the body shop to repair my wife’s car after a minor fender bender. That the officials in Wuhan were able to construct such a facility in so little time, without much time to plan ahead, speaks to both the creativity and resourcefulness of the architects, engineers, and laborers involved in this project.

Ordinarily, building such a facility would require years: clearing the land, laying the foundation, building out the structure, installing utilities, and finishing out the interior each require many months of planning and labor, and this work largely happens consecutively, not concurrently.

So this begs the question: If we can build such a facility in days rather than years, why don’t we always do it that way?

The answer, of course, is that a hospital designed to be built in 10 days is constructed with speed as the only consideration. Treating as many patients as possible, as quickly as possible, is the only goal. As a result, other attributes – quality, durability, maintainability, and comfort – are all ignored to satisfy the only metric that really matters in such a project: time. The interior photos show a facility that looks more like the inside of a U-Haul truck than a hospital. Outside, the exposed ductwork and skeleton-like walls reveal a structure that is unlikely to withstand the rigors of use.

As a data guy, I see this same pattern when building data architectures. Everyone involved in a data project wants to have a perfectly working data pipeline, with validated metrics and user-tested access tools, delivered at or under budget, and ready for use tomorrow. The challenge comes in when deadlines (whether legitimate or invented on the fly) become the only priority, and architects and developers are asked to sacrifice all else to meet a target date. Sure, you can add a lot of hands to the project, like they did by engaging 7,000 people to build the Wuhan hospital. Throwing more people at the problem might get you a solution more quickly, but the same shortcuts to sacrifice quality, durability, and maintainability will need to be made.

When setting schedules with my clients, I sometimes have to work through this same thought exercise. Yes, we could technically build a data warehouse in a week, but it’s going to be lacking in what one would normally expect of such a structure: many important features would be left out, it’ll likely be difficult to maintain, and there would be no room for customization of any type. And, like the temporary Wuhan hospital, it would likely be gone or abandoned in 18 months.

Building something with speed as the only metric is occasionally necessary, but only under the most extreme of circumstances. Creating a data architecture that delivers accuracy, performance, functionality, and durability requires time – time to design, time to develop, time to test, and time to make revisions. Don’t sacrifice quality for the sake of speed.

Join us in Consultant Corner at SQL Saturday Dallas

Do you have questions about business intelligence, analytics, Power BI, or data architecture? If so, we would love to chat with you at the SQL Saturday Dallas event this spring.

On June 1st of this year, we will be hosting a Consultant Corner at SQL Saturday Dallas. Consultant Corner is a casual space where you can have one-on-one conversations with data experts. If you have specific “how do I …?” questions, or if you are just looking for general advice about the business intelligence and analytics landscape, we would love to chat.

We are co-hosting this event with our friends over at 28twelve Consulting. Like us, they are focused on building outstanding solutions in the Microsoft stack, and are great at helping to navigate folks through the multifaceted world of business intelligence.

Registration for SQL Saturday Dallas is free, with an optional on-site lunch for $12. We will be set up in the Consultant Corner in the vendor area all day. We look forward to seeing you there!

Which Is More Secure: On-Prem or Cloud Data?

Which Is More Secure: On-Prem or Cloud Data?The short answer is this: Data, regardless of storage location or medium, is as secure as you make it.

The use of cloud-based data processing and storage has grown significantly in the past decade, a trend that is expected to continue. A recent IDG Enterprise survey reported that 70% of organizations surveyed had moved at least part of their data or infrastructure to the cloud, with another 16% planning to do so within the next year. Cloud data storage is no longer something that “other companies” do. The value in both cost savings and time-to-market means that most organizations can realize a benefit from moving at least some data and applications to the cloud.

When comparing cloud to on-prem, each has a number of benefits and shortcomings. On-prem data storage is inherently more flexible, because you can configure your environment exactly how you want it. Cloud data storage reduces up-front costs by eliminating some hardware purchases and data center build-out costs, and can often be configured in hours as opposed to weeks or months for on-premises storage. There is no one-size-fits-all solution; some apps will be a better fit for the cloud, while others work better with on-prem storage and processing.

Which Is More Secure: On-Prem or Cloud Data?

Of the reasons given to resist cloud data storage, by far the one I’ve heard the most is a concern over security. The traditional approach for applications and data storage has been to keep those things within the walls of the organization, especially when the nature of the data is especially sensitive (such as HIPAA or other PII data). I can’t count the number of times I’ve heard someone say that they’d never migrate a particular workload to the cloud because it isn’t secure enough.

I really do sympathize with the emotion of such a statement. Moving sensitive data from a location over which you have physical control (a data center) to one that you don’t (cloud storage) feels like you’re surrendering control of that data. For those unaccustomed to trusting an outside vendor to host their data, this can be an uncomfortable feeling.

However, to look at this in context, think about all of the ways your data is currently being used even when stored on-prem. The odds are high that one or more of these statements is true:

  • Some internal “super users” have carte blanche access to data
  • Some systems allow for generic password-based logins (such as using SQL logins rather than Windows authentication), permitting largely unaudited data access
  • Access to the database itself is controlled and audited, but the backups do not have the same level of security
  • Those responsible for monitoring your firewalls and other barriers also wear a lot of other hats, and/or do not provide 24x7x365 coverage for security monitoring
  • Backups of data are physically transported offsite
  • Service accounts that run core database services (such as SQL Server Agent) have broad permissions for data access for convenience
  • Some data extracts shared with outside vendors, who may also be permitted to share it with their vendors as needed
  • Some vendors even have direct access to your systems
  • Data from third parties is imported into your systems on a scheduled basis using automated tools
  • Nontechnical vendor staff (such as janitorial or building maintenance personnel) have physical access to locations where data is stored
  • Password length and/or complexity is not systemically enforced

This is a very small subset of an innumerable list of activities that require a good deal of trust in people, both internal and external. Data loss and data breaches are only as secure as the weakest link in the virtual chain, and it is far more likely to have a failure of either human error or system design than to have such an event occur simply because you chose a cloud platform versus storing data in-house. A bad guy will have a much easier time crafting a social engineering breach or dictionary attack than hacking the platform itself, whether on-prem or cloud.

Security Depends on You

This doesn’t mean you shouldn’t ask tough questions regarding security when selecting a cloud vendor. When evaluating those services, ask about topics like colocation, multitenancy, penetration testing and monitoring, and physical security. Ask how and when their vendors and partners would have access to your data. Apply at least as much scrutiny in this decision as you apply to other vendors who will have access to your data. But don’t assume that the data is not secure just because you don’t have physical control to the data center or media.

When choosing the cloud for application or data storage, you’re simply trusting a different team to handle what your in-house staff would manage for on-prem storage. In many cases, the cloud vendor will have deeper expertise, better proactive monitoring, and round-the-clock coverage that an in-house team may not be able to provide.

As a business owner, I trust my application and data storage needs to cloud solutions. Tyleris Data Solutions does not own any physical servers, instead relying on trustworthy cloud vendors to store our mission-critical data. There will be some workloads that I would want to handle in-house in a local colocation, but with respect to security, my vendors are far better equipped to manage round-the-clock security than my team is.

The fact that you can walk into your data center and physically touch the server hosting your on-prem data does not make that data more secure. Proximity does not imply security. Data and applications are only as secure as you make them, whether hosted in-house or in the cloud.