Devagnostic

In order to capture suspicious data from external sources, we usually define a set of rules, that explicitly examine upcoming data and validate the data against those rules. What happens if the data still looks good and stays within defined frames and schemas, but something is smelly? Classic approach Let's consider the case of a company that tracks the estate market changes. If the volume of the data that comes from an external data provider rumps down, or values break their banks, then it is easy to capture that breach by introducing validation rules. For example, If a price of an estate is bigger than 100M$ or lower than zero, then such input data (like a file) should be rejected or fixed, before processing. The business users maybe not be happy with some delay, but still ... it's better to be safe than sorry. Now let's consider a case when the average price of your property drifts during the time. If one week an average price of a property equals 100k$ and the next week...

Despite the fact I've spent the last two years working with Microsoft Azure cloud tech stack, I didn't have a chance working with Azure Synapse yet. Finally I've met this guy. The first impression is very good. Azure Synapse Workspace To create the first Azure Synapse you need to create an  Azure Synapse Workspace . If you are already familiar with Azure you may already know Azure Log Analitycs Workspace to examine logs and operation of your applications and resources. Here is the same. Workspace is created within the Azure resource group, building a landing page for its settings, analytic pools, security, and monitoring aspects. Azure Synapse Workspace will require a storage account to be created/selected under-the-hood. A storage account will be used to build Azure Data Lake Storage Gen2 for the files you will manage by yourself (sample data for processing) or by the Azure Synapse itself (ex. PySpark trained models will be saved here as well).  The better place to deal...

Finally is there! My first paper book has been printed out. The "Decommission Analysis Guide" is ready to be picked up from the Amazon bookshelf and read. If you have been asked to analyze tens (or more) systems in your organization,  and you're whole in tears now,  this book is for you. That's a practical guide, that will help you to rationalize the task, and embrace a massive number of existing systems you don't know and have no chance to understand. [ Amazon store ] What is system decommission analysis ? A life cycle of all IT systems ends up with plugging out of the power supply.  Before it happens, we need to make sure we understand what is the real scope of the system, and what actually happes once the last led-light goes out. It's not usually a big deal if we have a single system to analyze and an existing support/developer team  already that is up to speed, with an insight into the product.  The problem becomes more interesting once you realize you hav...

While preparing for the Azure exam, I've noticed a supportive initiative on Facebook to help a little girl collect money for her surgery.  A number of my colleagues have already participated in that action, so I've decided to join as well. The challenge was to prepare any "program" staying in 16 lines codes, sharing it publicly on the FB, and add the #code16challenge hashtag. Playing a bit recently with Azure, I've decided to use this opportunity to create a small web app   (nodej.js/npm), set up all required Azure resources ,   and deploy the app to the cloud  - all in 16 lines of code. web= "code16challenge-siepomaga" asp= "wa2021asp" rg= "code16challengeRg" npm init -y  # Webapp setup echo  $(jq '.scripts.start="node index.js"' package.json)  > package.json echo \ "const http = require('http');\  http.createServer((req, res) => {\   res.writeHead(200,...

Test case automation may bring many benefits for the application/system QA process, but it's not always obvious how to introduce an implementation of the test cases into SCRUM flow.  The SCRUM methodology defines the number of PBIs (product increments) for prioritization and planning, as a part of the SCRUM process. Those could be user stories or enablers , sometimes bugs . Each of the PBIs needs to be estimated and confirmed by a scrum team before it's added into a sprint scope (sprint backlog). The question would be: How to plan the effort for the implementation of automation tests, as those are not the PBIs per se? Test Case is not a PBI In the strategy proven in one of my projects, a test case is an output (product) of the delivery, just like an ordinary artifact. Typically, it's a part of a testing suite. It needs to be created, maintained, and placed into a repository (see: Azure DevOps ). Once prepared, we can start working on the automation of the test case. Bo...