Welcome, Joshua Ferguson!

Here we grow! Thanks to the numerous clients we have partnered with in the past year, Tyleris Data Solutions is expanding to add another skilled data architect to our team.

Joshua FergusonWe are proud to welcome Joshua Ferguson as the newest member of the Tyleris team. Joshua is a highly skilled technologist and a pragmatic problem solver, with a keen ability to bridge the gap between business needs and technical specifications. He studied informatics as an undergraduate, and later earned his Master’s Degree in Computer Science from Arizona State University.

Joshua has worked in various industries throughout his technical career, most recently having worked as a business intelligence architect at a healthcare company. He currently resides in Japan where his wife teaches English to second-language learners.

Joshua has already gotten plugged in to some exciting work with Tyleris clients, and you will likely see more from him both in our professional engagements as well as through our blog and on social media. We are delighted to have him on board!

Join us in Consultant Corner at SQL Saturday Dallas

Do you have questions about business intelligence, analytics, Power BI, or data architecture? If so, we would love to chat with you at the SQL Saturday Dallas event this spring.

On June 1st of this year, we will be hosting a Consultant Corner at SQL Saturday Dallas. Consultant Corner is a casual space where you can have one-on-one conversations with data experts. If you have specific “how do I …?” questions, or if you are just looking for general advice about the business intelligence and analytics landscape, we would love to chat.

We are co-hosting this event with our friends over at 28twelve Consulting. Like us, they are focused on building outstanding solutions in the Microsoft stack, and are great at helping to navigate folks through the multifaceted world of business intelligence.

Registration for SQL Saturday Dallas is free, with an optional on-site lunch for $12. We will be set up in the Consultant Corner in the vendor area all day. We look forward to seeing you there!

What To Look For When Hiring A Data Professional

What To Look For When Hiring A Data Professional

check_smFinding just the right data professional to hire is one of the most challenging tasks an organization can undertake. While hiring a team member for any role requires a great deal of work and care, the role of the data professional is particularly challenging to fill. From day 1, the data professional will have access to and responsibility over the company’s most valuable asset. These roles usually require a mix of hard skills and soft skills, and often require engagement with people at every level, from peers to executive leadership.

Finding the right person

Here at Tyleris Data Solutions, we are getting ready to grow our team this year. In preparation, I have been thinking a lot about the attributes that we should look for in our new team member. While there will always be a longer and more specific list of needs for each role, these are the attributes I have identified that I look for in every data professional.

Integrity. This one is first on the list for a reason, and is the one attribute where compromise is not acceptable. Data professionals have vast access to an organization’s data, and if that information were to be lost or stolen, it could literally end the company. The thing about integrity is that it is almost impossible to fully assess in an interview. Learning about a person’s level of integrity takes time and effort, which is why hiring a data professional should be a slow process.

Intellectual curiosity. Among all of the technical professionals I’ve worked with, I’ve learned that those with a strong intellectual curiosity tend to be more effective. Team members with this attribute often go out of their way to learn about other areas of the business or technical architecture that aren’t necessarily required for the job, leading to a better big-picture view of how the organization uses data.

A positive and empathetic attitude. Increasingly, data professionals have highly visible roles, requiring them to engage with peers, superiors, customers, and clients. Their attitude is the backdrop for each one of those interactions, so it is essential that the data professional come to the table in the right frame of mind. Having an empathy for one’s constituents will improve the quality of the job one performs.

Technical aptitude. The data field is rapidly evolving, and requires of data professionals the willingness and ability to quickly learn new things. Hiring staff members with technical aptitude will help to build a team that is adaptable and can assimilate into new technologies quickly.

Initiative. There are folks who wait to be told exactly what to do, and others who go figure out what needs to be done and then do it. Not every team member has to have this go-getter attitude, but each team needs at least a few people with this characteristic.

Experience. I put this one at the bottom of the list for a reason. It’s not that experience isn’t important – it is! – but of all the items on this list, experience is the one thing that the organization can give to the team member after they are hired. A person with minimal experience but who possesses all of the other attributes on this list is going to be a very compelling candidate.

Hiring is hard. Hiring technical professionals is especially challenging, and is critical to get right. While technical skills are important, finding the person with integrity, attitude, and aptitude will help to build a solid team.

This post was originally published in my Data Geek Newsletter.

Our Relationship with Facebook

At Tyleris Data Solutions, we are data people, and by extension, our first and primary role is that of data stewards. With each and every one of our relationships, our overriding concern beyond all other tasks is the security and privacy of data. In the partnerships that we build with other companies, we look for a similar level of care and concern around protecting the data.

Since our inception, we have used Facebook, both as a social media platform as well as an advertising outlet. During the past year, we have become aware of a number of serious security and privacy issues around Facebook’s protection of and use of data. We strive to only do business with organizations whose partnerships reflect well on us, and vice versa. Based on what the data breaches and the privacy decisions within Facebook, we feel that we can no longer engage with them in any capacity.

Starting today, we will no longer be updating or monitoring our Facebook page, nor will we be responding to any messages sent through Facebook Messenger on that page. In addition, we will discontinue indefinitely all advertising on Facebook.

We are still available for our clients and followers on our website, our newsletter, or by telephone at 214/509-6570. We are also on Twitter, and have recently established a presence on MeWe, a promising new social media platform that is very focused on data privacy.

As always, thanks for your business and for your attention. Feel free to contact us with any questions.

Webinar: Getting Started with Change Tracking in SQL Server

Change TrackingStart your summer off right by brushing up on a highly effective change detection technique! We will be hosting a webinar, Getting Started with Change Tracking in SQL Server, on Friday, June 8th at 11:00am CDT.

In this webinar, I’ll walk you through the essentials of change tracking in SQL Server: what it is, why it’s important, and how it fits into your data movement strategy. I’ll walk through demos to give you realistic examples of how to use change tracking.

Registration is free and is open now. I hope to see you there!

Famous Last Words: “That Should Never Happen”

That should never happen, right?When it comes to managing data, there is often a difference between what should happen and what can happen. That space between should and can is often challenging, forcing data professionals to balance risk with business value. A couple of common examples:

“There should not be any sales transactions without a valid customer, but because the OLTP system doesn’t use foreign keys, it could theoretically happen.”

“Location IDs should be less than 20 characters, but those aren’t curated so they could exceed 20 characters.”

“This list of product IDs from System A should match those in System B, but because they are entered manually it is possible to have a few typos.”

“This data warehouse should only contain data loaded as part of our verified ETL process, but since our entire data team has read-write permission on the data warehouse database, it’s possible that manual data imports or transformations can be done.”

“This field in the source system should always contain a date, but the data type is set to ASCII text so it might be possible to find other data in there.”

“The front-end application should validate user input, but since it’s a vendor application we don’t know for certain that it does.”

Managing data and the processes that move data from one system to another requires careful attention to the data safeguards and the things that can happen during input, storage, and movement. As a consultant, I spend a lot of time in design sessions with clients, discussing where data comes from, how (if at all) it is curated in those source systems, and what protections should be built into the process to ensure data integrity. In that role, I’ve had this conversation, almost verbatim, on dozens of occasions:

Me: “Is it possible that <data entity> might not actually be <some data attribute>?”

Client: “No, that should never happen.”

Me: “I understand that it shouldn’t. But could it?”

Client (after a long silence): “Well…. maybe.”

Building robust systems requires planning not just for what should happen, but for what could happen. Source systems may not include referential integrity to avoid situations that are impossible in business but technically possible inside the data store. Fields that appear to store one type of data might be structured as a more generic type, such as text. Data that should be in curated lists can sometimes contain unvalidated user input. None of these things should happen, but they do. When designing a data model or ETL process, be sure that you’re asking questions about what protections in place to make sure that the things that shouldn’t happen, don’t.

Flexible Consistency

Flexible ConsistencyDuring last week’s PASS Summit day 2 keynote, Rimma Nehme discussed some off the architecture behind Cosmos DB, the globally-distributed flexible database system rolled out earlier this year. Dr. Nehme’s presentations are almost always buzz-worthy, with a great mix of technical depth and big-picture vision. Take 90 minutes to watch the recording; you won’t be disappointed.

Among the many notable points in this presentation was a journey through the concept of database consistency. Database consistency, also referred to as transactional consistency, is the attribute of database systems describing the coupling of different parts of a single transaction.

The simplest example of this is the data tasks that go into a banking transfer. In the database, money doesn’t “move” from one account to another; rather, the amount of money in one account is decreased by adding a credit line item, and the same amount is added to the other account by inserting a debit line item. Although these are two separate operations, they should be treated as one, with the whole operation succeeding or failing as one transaction. In a transactionally consistent system, the two parts of the transaction would not be visible unless both of them were complete (so you wouldn’t be able to query the accounts at the moment of transfer to see that money had been removed from one but not yet added to the other).

During Dr. Nehme’s keynote, she pointed out that most of us look at database consistency as a binary choice: either you have full transactional consistency or you don’t. However, data workloads aren’t always that simple. In Cosmos DB, there are actually five different consistency models to choose from.

Flexible Consistency

For those of us who have spent most of our careers working with ACID-compliant operations, having this kind of flexibility in database consistency requires a shift in thinking. There are many business cases where such transactional consistency is not critical, and the rise of NoSQL-type systems has demonstrated that, for some workloads, transactional consistency is not worth the latency it requires.

Some of us old-school data folks (this guy included) have in the past poked fun at other consistency models, using the term “eventually consistent” as a punch line rather than a real consideration. However, it’s no longer safe to assume that every database system must have absolute transactional consistency. As workloads become larger and more widely distributed, the use of less-restrictive consistency settings will be essential for both performance and availability.

This certainly doesn’t mean that transactionally consistent systems are going away. It just means that the classic RDBMS way of maintaining consistency is no longer the only way to do things.

Delivering Trustworthy Data and Analytics

Delivering Trustworthy Reporting and AnalyticsOrganizations of every size and in every vertical have a vested interest in unlocking the power of data. Among other benefits, reliable and timely data delivery will reveal what business processes are (or aren’t) working well, help understand customers’ motivations and behavior, and make internal workflows faster and more efficient.

Harnessing and arranging the data is only half the battle, however. Once it’s in your hands, you also need to transform it into a digestible report with actionable insights that audiences of every technical background can parse. Just as importantly, this system of reporting and analytics must be trusted to be reliable and accurate.

Delivering Trustworthy Data and Analytics

Unfortunately, problems with data reporting plague even the most well-prepared organization. In a recent survey, 73 percent of respondents indicated that they had to wait days or weeks for the insights they required, while almost 70 percent of them frequently encountered questions that their current analytics setup had no response for. Even worse, another survey revealed that only one third of executives surveyed trust the data and analytics generated from their business operations.

Building trust in the data is critical for the success of any information delivery project. If the data is inaccurate, difficult to understand, or improperly shaped for the task at hand, the consumers of the data will lose trust in the data. When that trust is lost, users will find other ways to get the data they need (often by building their own offline data store from other sources).

Here’s a look at five key actions that you need to take in order to start creating reports that you (and they) can believe in.

Establish the Source of Truth

A symptom of an immature information delivery strategy is that there is no single and clear source of truth. This fact cannot be overstated: Unless there is a single trusted source of truth for every key business metric, your information delivery architecture cannot be trusted to deliver accurate, validated data. Defining the gold standard for each measure, KPI, and reference list is a prerequisite for any data delivery initiative. Without this, you’re just spinning your wheels rather than making real progress.

Once the source and/or formula for each key piece of information has been defined, communicating that to the wider audience of data consumers is essential. Just as it is important to define each term and definition, you must include in that messaging which sources are to be used to validate any derivative reports, dashboards, or manual queries. Lacking that, your reports will tell conflicting stories.

Get User Input

You can’t deliver high-quality reports or dashboards without understanding what it is that your users need from the data. Getting user input is critical to the process, but all too often this necessary part of the project is shortened (or skipped entirely) to save time. It’s not enough to try to infer requirements from existing reports or dashboards; spend time with those who will be consuming the data to fully understand their needs.

When gathering user input for data delivery requirements, you must understand the following:

  • How do you use this data? Gathering input for information delivery requirements is a business exercise, not a technical one. When you begin gathering input from data consumers, seek first to understand how data consumption helps them do their jobs. With a clearer understanding of their business processes, you’ll be better equipped to deliver a solution that solves their business problem.
  • In the current system (reports, dashboards, etc.) for data delivery, what is working well? What is lacking? It’s important to learn from successes and failures of the past. Invest the time needed to understand what is currently being used to satisfy data delivery needs, and how well those components are working.
  • What are the business rules? Business rules describe relationships, data quality processes, exception and anomaly handling, and other structures and triggers that help turn raw data into actionable information. It is important to understand what business rules exist (or should exist) on the data, and at what point in the data lineage they should reside. Be aware, though, that non-technical users may be turned off by the term “business rules”. Instead, ask about data exceptions, manual clean-up tasks, and other open-ended topics that can reveal what business rules should be included in report delivery.

Make Things Clear

Context is key when it comes to data analytics. You wouldn’t give your users the number “8” without somehow contextualizing that figure–does it refer to the number of hours, the percent of hourly utilization, or one of a million other possibilities? Is that number good or bad? From where does that number originate?

Any data delivery solution must be built with clarity and transparency. For each data source, make clear its intended meaning, its origin and lineage, as well as any processing or transformation logic that has taken place. By providing this contextual information, you can help users understand the reliability of any given datum and find the source of errors or outliers.

Define Everything

Just as you provide precise and accurate data to your users, the language that you use to describe that information needs to be equally clear. Before you begin analytics and reporting, agree upon the terms that you use to measure outcomes, including metrics and key performance indicators (KPIs).

Even the most fundamental metrics should be defined. Something as basic as “a day” might be assumed to be a 24-hour period, but this definition can vary. For any essential metric, KPI, or descriptive term, be clear about what each represents.

Document Changes

Over time, data delivery solutions will evolve. The business needs will change, processes will get more mature, and occasionally the shape or grain of the underlying data will be modified. To maintain trust in your reporting or dashboard solution, be clear and proactive about any changes.

One of the worst outcomes for such a solution is for the data consumers to abandon it because they do not get consistent answers. Let there be no surprises for your reporting audience when critical changes need to be made. Communicating these changes and their impact requires extra time and effort, but that investment will help protect the trust in the data.

Cloud Adoption Concerns (And How To Overcome Them)

Cloud Adoption ConcernsCloud technology has become very pervasive, with 89 percent of organizations globally identifying it as part of their IT strategies while another 75 percent say they have already implemented at least one cloud application. Some of the key benefits enjoyed by organizations that adopt cloud technology include lower costs, faster time-to-market, improved reliability, and better business intelligence insights, among others.

In spite of the rise in usage, there is still a significant amount of concern and push-back for any potential new cloud-based solution. These include security concerns, integration challenges, industry standards compliance issues, information governance, and difficulty in measuring the real economic value of cloud solutions. In many cases, however, organizations often think these challenges are much bigger than they really are.

Cloud Adoption Concerns (And How To Overcome Them)

Here are five of the most common cloud adoption challenges and how to overcome them:

1. Security Concerns

Problem: Although security is becoming less of a concern for cloud adoption, there are still those who believe that data stored in the cloud is inherently less secure than on-prem data storage.

Solution: Your approach to security for your cloud workloads should not be that different from how you approach security for your on-premises workloads. Develop overarching security policies that are based on your particular needs within your organization, then use these as yardsticks to evaluate different cloud solution providers. Major cloud providers such as better than what a small or mid-sized organization can develop internally.

2. Integration with Existing Systems

Problem: It’s rare to migrate all of an organization’s infrastructure to the cloud at once, so there is naturally some concern over how migrated cloud applications will integrate with existing on-prem systems.

Solution: While cloud vendors were initially known for pushing an “all-cloud” approach, the industry has shifted to now support and adopt a hybrid on-prem/cloud model as the default architecture. As part of that shift, the major cloud vendors have made the process of integrating from on-prem to cloud (and cloud to cloud) much easier, through APIs, specialized integration frameworks, and hybrid authentication architectures.

3. Cost Concerns

Problem: Making a long-term commitment for services (rather than one-time hardware purchases) means that we’re locked in to a monthly subscription fee. It’s harder to predict what we’ll spend.

Solution: Organizations need to understand that transition and implementation costs are not unique to the cloud. You need to treat your cloud implementation just like you would a normal IT project regardless of whether it’s SaaS, IaaS or PaaS.

Remember that running your own data center isn’t a one-time capital investment. Servers and supporting equipment have ongoing costs as well, including electricity, cooling, physical security, and labor costs to install, maintain, and upgrade the hardware. Cloud models typically involve smaller one-time capital expenditures than on-premises models, but this model also allows you to scale up your architecture needs (and costs) as your business to grows. The cloud pricing model means you’re paying what you need today, without overbuying to anticipate what you might need tomorrow.

4. Concerns About Loss of Control

Problem: Organizations with on-prem systems often feel they are giving up too much control to cloud providers, surrendering the ability to define granular settings for system resources, networking, and security.

Solution: The cloud isn’t just one thing. One of the best analogies of the different cloud offerings is the pizza-as-a-service model. You can make your own pizza at home, buy a take-and-bake pizza, have a pie delivered, or go out to eat at a pizza place. These different services, much like cloud architectures, allow you to pick the level of service that meets your needs. For workloads that require the organization to have more granular control over how the application runs, the IaaS model makes it possible to migrate to a virtual machine in the cloud without giving up the ability to configure the machine environment, storage, etc. For other workloads requiring less hands-on configuration, using a managed PaaS model makes more sense.

5. Legal and Compliance Concerns

Problem: A common challenge to cloud adoption is that it may not meet compliance or regulatory demands.

Solution: The major cloud vendors are certified for use for most any regulated domain of data. Further, most vendors allow you to specify the geographic region(s) in which your application and data will be stored, thus eliminating the concern of violating geographic boundaries of compliance or regulation.

Conclusion

With cloud architecture becoming more and more common, most of the classic objections to migrating to the cloud no longer hold water. Security, integration, and compliance used to be blockers to cloud adoption, but the platforms have matured to the point that these are no longer barriers.