Devagnostic

While exploring the problem of migration to the cloud I've realized that it would be worth putting some light on the data owning issue. This is when the Azure Purview comes in. I'm not a Chief Data Officer, but I if had been the one, would have been asked some fundamental questions about my data - what is the source, how the ETL/ELT process looked like, who own the data ... and, what is the data linage..., and finally what all those data mean.  All those questions are very important these days. Those might be triggered by regulatory requirements (data lineage, anonymisation), data scientists implementations (Where is my source dataset used to train the model?), internal auditors or the business itself, who struggles with defining the common domain models for years. By working for one of my client's I've faced the problem of defining the Product Master application, which would be a golden source of domain model definitions across the company. The issue was too big to be ...

In order to capture suspicious data from external sources, we usually define a set of rules, that explicitly examine upcoming data and validate the data against those rules. What happens if the data still looks good and stays within defined frames and schemas, but something is smelly? Classic approach Let's consider the case of a company that tracks the estate market changes. If the volume of the data that comes from an external data provider rumps down, or values break their banks, then it is easy to capture that breach by introducing validation rules. For example, If a price of an estate is bigger than 100M$ or lower than zero, then such input data (like a file) should be rejected or fixed, before processing. The business users maybe not be happy with some delay, but still ... it's better to be safe than sorry. Now let's consider a case when the average price of your property drifts during the time. If one week an average price of a property equals 100k$ and the next week...

Finally is there! My first paper book has been printed out. The "Decommission Analysis Guide" is ready to be picked up from the Amazon bookshelf and read. If you have been asked to analyze tens (or more) systems in your organization,  and you're whole in tears now,  this book is for you. That's a practical guide, that will help you to rationalize the task, and embrace a massive number of existing systems you don't know and have no chance to understand. [ Amazon store ] What is system decommission analysis ? A life cycle of all IT systems ends up with plugging out of the power supply.  Before it happens, we need to make sure we understand what is the real scope of the system, and what actually happes once the last led-light goes out. It's not usually a big deal if we have a single system to analyze and an existing support/developer team  already that is up to speed, with an insight into the product.  The problem becomes more interesting once you realize you hav...