Pretty time

This week I’ve been working on another new cube which had the time spent, which is in minutes. To pretty it up it wanted to be in days, mins and hours.

Here is a simplified version of the final output

prettytime

and the hopefully easier to understand MDX code

Adding the days option isn’t much more work – it does however make the code look alot more angry. Also you have the issue of what is a day – is it 24 hours? Is it the working hours, or is it less, is it the realistic billable time after you deduce admin time?

Dangers of distribution lists

Distribution list are amazing things, they allow us to send mail to our team without having to remember who exactly is in our team, this is especial true in this modern age where we are part of many different teams, sometimes without even realising. These lists can contain hundreds or even thousands of members and the latest NHS IT bug has left IT departments maybe wanting to double-check their distribution lists – hopefully before the security team comes a knocking.

Luckily, this is really easy in Excel 2016. Built into Excel 2016 is PowerQuery and one of the out-of-box connectors is Active Directory. With a bit of PowerQuery magic, you can easily pull this data out.

So we want to:

  • Connect to our Active Directory
  • List out all our Groups
  • Filter to email-able Groups
  • Include
    • the group owner
    • the members
    • any security \ restrictions on who can email the list

Note: If your unluckily enough to be running an old version of Excel, it possible to do the same thing in Power BI Desktop, which is a free download.

Below is the M code to do this:

For more help on how to enter the M code into Excel see my previous post.

I’ll update this post later with a few useful DAX calculations and hopefully a useful template (both Excel file and a sexy PowerBI report) – the later will take a while as I have to build a test VM.

Comments

One of the annoying things about working with multiple languages is they each have their differences – obviously, otherwise there would only be 1 language! Although there is good reason to have these differences they still have elements that are a pain. One of the most annoying things, other then “do you require a semicolon at the end of the statement?” is comments. Comments aren’t executed or complied but they help developers read code. Remember, the code is a interpretive language, its designed to be gateway between the machines and humans, if you write something complex your more then likely need to be include more comments explain what it does or look at refactoring so other developers can continue your work.

The few options are:

Double dash (–)

— Single line quote

Single Hash (#)

# Single line quote

Double forward-slash (//)

// string domain = “matt40k.uk”;

All of these are single line quotes, as in each line needs the require comment and the comment continues till the end of the line \ carriage break.

You also got:

Forward-slash and asterisk (/*)

/* Multi-Line
quote */

Forward-slash and asterisk marks (/*) the start of the comment, the comment continues until the asterisk marks and Backward-slash (*/). The code must close any open Forward-slash and asterisk marks (/*) comments in order to compile.

With Forward-slash and asterisk, you can comment in-line, for example

Select * from /* Change the code to the right */ sys.tables

This is valid T-SQL that will return the all the columns for meta data about tables on the selected database

You can also use this across multiple lines, for example

select

TableSchema = schema_name(schema_id)

,TableName = name

/*
You could also add

other tables

 

*/
from

sys.tables

 

Power Query Formula (M) Language

Double dash (–) No
Single hash (#) Yes
Double forward-slash (//) No
Forward-slash and asterisk (/*) Yes

Transact-SQL (TSQL)

Double dash (–) Yes
Single hash (#) No
Double forward-slash (//) No
Forward-slash and asterisk (/*) Yes

C sharp (C#)

Double dash (–) No
Single hash (#) No
Double forward-slash (//) Yes
Forward-slash and asterisk (/*) Yes

 

Fun fact, my most commonly used comment?

To-Do

Where has SIMS Bulk Import gone?

Cut-down version, I’m not working with SIMS anymore and I can’t pass on my work without them getting a large bill from Capita for using the business objects.

Years and years ago I was working on a SIMS help desk at a Local Authority (LA). I had a school log a call asking about importing email addresses. Like many schools this school had just purchased a messaging service that allows sending emails and texts, the problem was they needed them in SIMS .net, as you can imagine the idea of manually retyping 1,000+ email addresses was rather daunting. So, after putting a call into Capita Partner Team asking for the API documentation I spend the evening building a simple import process. The next day I phoned the school to give them the good news and we imported their email addresses. A few days later a few more schools asked the same question and I decided to continue spending my evenings expanding (it imports email, telephone and User defined fields from XML, CSV and Excel spreadsheets) and refining the tool. I’ve always developed in my own time, I’ve never claimed overtime etc and I’ve never charged a penny for using it.

SIMS Bulk Import uses the SIMS .net business objects (API), its uses the the SIMS .net DLLs. Its the exact same .NET libraries that the SIMS .net library uses.

At the time we were in a partnership with BT, who had a procedure for raising new business ideas. The process involved working out what type of change it was – in this case it was a efficiency saving. To put it simply this means you can’t charge more then what you can save, in this case labour.

Capita charge the school. Then they charge the partner for write access, which the partner then charges on the school so the schools end up paying twice for support on one thing.

When you take into account the Capita charge, the handling of money costs (invoicing, collecting and chasing etc), the helpdesk costs, the fact people expect a certain standard, ie you’re going to have to invest alot more in terms of development – including documentation. It just doesn’t make sense. It’s actually more cost effective to hire a temp to manually key in all those details!! Nuts!!

So at this point I basically decide I’ll give it away. I really didn’t want to see my hard work go to waste. I basically managed to wangle it into the public domain without finding a massive Capita bill land on my desk!! Its been in a wild for many, many years (with Capita knowledge) with a grand total of ZERO corrupt SIMS databases. I find this quite an achievement. Don’t get me wrong, SIMS Bulk Import has failed a number of times, but it’s never left your SIMS system in a worse state (unless you’ve done something stupid like successfully imported the same email address to every pupil in SIMS!)

A few years later I switched teams and stopped working with SIMS .net. SIMS Bulk Import has been stable for a while and I’ve had a few commits from individuals. I’m now at the stage where I’m going to leave the LA and go work somewhere else and its unlikely to have a SIMS .net license let alone API access. I needed to find a new owner for SIMS Bulk Import. Anyone who’s talked to me would have described SIMS Bulk Import as the poor man’s Salamander Active Directory, it is simply put the next logical step if you work out how you’d improve SIMS Bulk Import, its what I would have done to make SIMS Bulk Import into a commercial product. Luckily Richard agreed to take it on and even help me recover some of the costs of SIMS Bulk Import. Before you shoot off to SalamanderSoft to download it, let me save you the disappointment. Capita has said they would charge them a license fee for each school using it, ie it wouldn’t be free. At this point I guess you can see where I’m going with this? SIMS Bulk Import isn’t worth paying for and Richard already has the expanded version (that IS worth paying for).

So in short, your options are:

  1. Except you can’t bulk import anymore and get typing or copy and pasting \ hire a temp
  2. Look at automating your processes and buy Salamander Active Directory
  3. Wait for Capita to come out with their own product – I suspect they will and charge. They do a limited SQL script that injects the records directly into the database, ironically bypassing the business objects (but hey, they support it so its all good right?)
  4. Switch MIS who doesn’t charge for partner access \ gives you bulk import routines (Eduware Network has a list of MIS suppliers)

Option 5 is carry on using it. I’m sure even with me saying, no, don’t do that (and I’m sure Capita will agree) – someone will. So a few comments.

Make sure you have the latest (or should that be last) version – its 2.5.0

It should be digital signed, think of it as SSL for your applications. If you right-click on SIMSBulkImport.exe or the .msi installer you should see an extra tab – Digital Signature and you should see its signed by me – Open Source Developer, Matt40k. If your copy doesn’t have the signature its possible code has been injected and it is unsafe.

digitalsignature

You should be OK as it uses whatever version of SIMS API you have installed, so it’ll just break one day and by break I mean it won’t let you login or will just give you all import failed (if it does fail in terms of SIMS database corrupt then some thing terrible has happened with Capita API, but I digress).

A few people have forked my code, for whatever reasons, I would just point out that the most up-to-date fork is 80 commits behind mine. That’s a fair amount of work that missing from those forks.

Anyway, hopefully you’ve found it useful whilst it last.

Are hybrid shops missing the point?

With rise of the internet high street businesses are rushing to adapt so they don’t become a thing of the past. However in their rush are they being a storm trooper? Are they missing the point?

First, lets look at two successful hybrid business. Hughes, a electrical store based in East Anglia and Argos, a British catalogue retailer operating in the United Kingdom. If we focus on Argos as its more well known business.

If you order online you can see the in store stock. You can then go in store and collect it. High street shops have become showrooms and mini warehouses.

Compare this to Game and Staples, these stores compete against each other. Online vs instore. The prices are different, online prices are cheaper. When you order online to collect in store, they post your order to the store. You order comes from the central stock, it can’t come from your local store.

It breeds resentment between local stores and corporate HQ. It leads to poor customer service.

I recently cancelled an online order with Staples because they claimed to do next-day delivery, I arrived in the store the next day only to find my order hadn’t arrived. So I phoned up, cancelled it and bought it instore. So it cost them

  • to answer my phone call, because I couldn’t cancel via the website
  • to post my order
  • to receive my order, instore
  • to post back my order to their central warehouse
  • to answer my phone call chasing the refund
  • to answer my phone call chasing the refund, again
  • to answer my tweet, again chasing the refund
  • to then refund my money (not sure if they have to pay a card charge)

All these actions have a staff cost, after all, staff don’t work for free.

Missing

SQLPrompt

So one of the really nice sponsors at SQL Saturday had a vending machine that was rigged up to release a treat when you sent a tweet. Like so

and give you…

In mine had a SQL Prompt license. At first I was a bit disappointed, I hoping for SQL Test license or a full blown SQL toolbelt but the more I’ve been using it, the more I’m loving it.

Like the IntelliSense… or the fact press table next to the * expands it out into the list of columns

sqlprompt_1

The fact it re-opens the tables you had open the last time you were in SQL Management Studio, that it has a history. I’d like to say I never close the wrong window, especially when I haven’t saved them, but I do. Luckily with SQL Prompt it allows me reopen it.

sqlprompt_2

It even stops DBAs going to jail :p

 

 

Wooo!!! Fixed in SQL Server 2016!

https://twitter.com/matt40k/710468378050339000

It’s nice to see Microsoft fixing old bugs – the I noticed today they updated the MSDN article about Actions in Multidimensional Models.

SSAS supports HTTPS actions

It now supports HTTPS!

Woot!!

https://connect.microsoft.com/SQLServer/feedback/details/692837/cannot-specify-https-in-an-analysis-services-report-action

 

 

 

Another year, another SQL Saturday

Unfortunately this year I’ve not been able to make any of the pre-cons, however the (free) community day looks epic – the hardest part is going to be selecting which session to attend, the final one looking the worst. Aghhh!!! I’m going to have to roll a dice or something!

CDC – Change Data Capture in Detail
Uwe Ricken

What’s new in the SQL Server 2016 for a BI Professional
Prathy Kamasani

Exploring the Azure SQL Database
Murilo Miranda

All things time-related
Kennie Pontoppidan

Data-driven personal decisions with PowerBI and Machine Learning
Ruben Pertusa Lopez

One session that really peak my inter-geek, which had me looking at getting a Microsoft Band – only to find out they are running the stocks low at the moment (they release the first and second one in October, so it looks like the 3 will be out soon).

The (Consumer) Internet of Things on your arm – MS Band & PowerBI connected

The Internet of Things (IOT) gets more and more attraction - not only on the business but also on the customer side. Connected fridges, cars and smart watches - always and everywhere connected! In this session Wolfgang will show you some possibilities of the Microsoft Band 2 SDK: how-to connect and read sensor data out of this device. But what should be done with that data? 
Power BI seems to be an ideal candidate for analyzing and presenting those kind of data. The different types of real-time analytics (Stream Analytics, Power BI API, ..) will be presented and their pros and cons will be envisioned. 
The challenge: Let's prepare a real-time dashboard of Band2 data in Power BI in 60 minutes!

Wolfgang Strasser

It’s also good to see the a small local company getting heavily involved :p

And not forgetting the Micro:Bit sessions on

Imagine If – Train the Trainer Micro:bit session

Want to help inspire young people and deliver a Micro:bit workshop?

If you are a teacher, parent or anyone interested in inspiring the next generation of technical experts; then come along to the Micro:bit session to get you started on how you could deliver a micro:bit workshop for either Primary or Secondary school children to code and create. 

In this session you will be taken through a workshop setup and approach (introduction, 3 coding tasks and a quiz) as well as getting hands on with the BBC micro:bit

This session requires external registration via http://microbit_opt1.eventbrite.co.uk

Amy Nicholson

Care Quality Commission (CQC)

Today one of our older SSIS packages failed which loads data into our warehouse, it turns out one of the fields had been expanded as the business had started to use a different standard for one of the codes. Despite not being told about this, it causing it fail and generate extra work on an already busy Monday it was actually a really good thing.

Firstly, once we identified the problem (and fixed it), our main business contact was aware of it, was able to explain why it was happening. The change in a nutshell was to change from using a internal-only generated code to using the national standard – Care Quality Commission (CQC) ID.

Part of the next step of the change will be to bring on-board the CQC data which include the Care Home ratings. This will create a unified view of placements and ensuring no-one is in Care Home rated as inadequate.

CQC provides the data either by CSV\Spreadsheet or an API.

Initial I thought importing the data dumps would be a good way forward, however after playing with the API, it looks like this might be a much better way forward. Below is a PowerShell script I knocked up to test the API out.

 

It’s really good to see that APIs are being generated, its just a shame these aren’t being embedded directly into the software. Still, its a long journey, but at least its started.