Thursday, May 24, 2018

Should algorithms be in the driver's seat?

A few days ago I wrote about going to the AI Talks meetup and disagreed with the view that humans should always have the last word. I just read “Should algorithms be in the driver's seat?” on Daniel Kahneman’s opinion on the subject, and we share the same views regarding biases affecting humans (in truth, my own view was probably heavily influenced by his book):

“The big advantage that algorithms have over humans is that they don’t have noise,” Kahneman said. “You present them the same problem twice, and you get the same output. That’s just not true of people.”
A human can still provide valuable inputs that a machine can’t — impressions, judgments — but they’re not particularly good at integrating information in a reliable and robust way, Kahneman said. The primary function of a person in a human-machine relationship should be to provide a fail-safe in an emergency situation. […]

You can combine humans and machines, provided that machines have the last word,” Kahneman said. He added: “In general, if you allow people to override algorithms you lose validity, because they override too often and they override on the basis of their impressions, which are biased and inaccurate and noisy.

I do think Kahneman here is talking about Narrow AI (*) and not the General AI about which concerns have been raised, but it’s good to read someone actually remind us that, Wait a minute, we’re not perfect, maybe lets turn the priorities around.

The article above includes the full video of a conversation at the 2018 Digital Economy Conference in April this year in NYC, where he made these coments.

(*) Saying Artificial Narrow Intelligence, although correct, just sounds awkward to me.

Sunday, May 20, 2018

Truly intelligent machines

This interview with Judea Pearl in Quanta Magazine apropos his book “The Book of Why: The New Science of Cause and Effect” has been making the rounds on social networks . Here are some thoughts:
“[…] as Pearl sees it, the field of AI got mired in probabilistic associations. These days, headlines tout the latest breakthroughs in machine learning and neural networks. We read about computers that can master ancient games and drive cars. Pearl is underwhelmed. As he sees it, the state of the art in artificial intelligence today is merely a souped-up version of what machines could already do a generation ago: find hidden regularities in a large set of data. “All the impressive achievements of deep learning amount to just curve fitting,” he said recently.”
It is true that the hype around ML/DS is at a peak. The stream of new posts on Twitter and LinkedIn on algorithms like Gradient Descent or their applications/pitfalls give me new things to read every day. I have downloaded a few free Machine Learning books a couple of weeks ago, and what is notorious is that they jump very quickly into math and statistics, from linear algebra to derivatives and calculus. A few days ago my partner was doing something with Deep Neural Networks, and I suddenly saw her get paper and pencil and start solving derivatives. “What are you doing?” “Oh, I have to solve this to implement back propagation”…
I had one class of Artificial Intelligence when I did my Computer Science degree way back then. We studied things like A* (the algorithm used to explore and rank alternatives in games like checkers), Expert Systems to help diagnose health problems, SNePS for knowledge representation and talked about Neural Networks briefly. There was no complex maths or statistics. Now, I’m not saying that “those were the good old days” - as the area was having one of its “AI Winters” and finding limited success, but stating that today’s conversation is dominated by a relatively narrow set of Narrow AI techniques, heavily based in statistics and focused on training models with very high volumes of data. These are proving successful and having a major impact in many areas, with more to come. But is this it? Will this be what leads us to AI, is this THE critical approach that will crack [General] AI, or will we bump into a “local maximum”?
“I can give you an example. All the machine-learning work that we see today is conducted in diagnostic mode — say, labelling objects as “cat” or “tiger.” They don’t care about intervention; they just want to recognize an object and to predict how it’s going to evolve in time.
I felt an apostate when I developed powerful tools for prediction and diagnosis knowing already that this is merely the tip of human intelligence. If we want machines to reason about interventions (“What if we ban cigarettes?”) and introspection (“What if I had finished high school?”), we must invoke causal models. Associations are not enough […]”
This seems to be a powerful argument. But where Judea Pearl gets really challenging is with the following:
“As much as I look into what’s being done with deep learning, I see they’re all stuck there on the level of associations. Curve fitting. That sounds like sacrilege, to say that all the impressive achievements of deep learning amount to just fitting a curve to data. From the point of view of the mathematical hierarchy, no matter how skillfully you manipulate the data and what you read into the data when you manipulate it, it’s still a curve-fitting exercise, albeit complex and nontrivial. […]
I’m very impressed, because we did not expect that so many problems could be solved by pure curve fitting. It turns out they can. But I’m asking about the future — what next? Can you have a robot scientist that would plan an experiment and find new answers to pending scientific questions?
The first part of this will no doubt create aversion in most people – myself included – who are passionate about where tech is going, but it’s hard to shake off the feeling – especially with the second part – that he probably has a point here. It may not matter for the people and companies, as the achievements are indeed impressive, and all across areas like healthcare, retail, autonomous driving, finance – or anywhere with data to process – there is a wealth of data to process and new automation/personalization to do. The impact in our lives will continue to happen. But what we’re doing at the moment is likely not enough.
And just one more quote from this interview.
“AI is currently split. First, there are those who are intoxicated by the success of machine learning and deep learning and neural nets. They don’t understand what I’m talking about. They want to continue to fit curves. But when you talk to people who have done any work in AI outside statistical learning, they get it immediately. I have read several papers written in the past two months about the limitations of machine learning.
[…] a serious soul-searching effort [is developing] that involves asking: Where are we going? What’s the next step?
Maybe the momentum, the exponential speed of development in the field, will make this question moot. Something will will come up – or many somethings - that will allow the tech companies to go on. Or maybe we just set a bar so high that, even though these data-based approaches can perform better than humans in more and more areas, we deem them insufficient because “self-driving cars using similar technology run into pedestrians and posts and we wonder whether they can ever be trustworthy.“ (WSJ, paywall, article on the same topic)

Thursday, May 17, 2018

AITalks: How can our digital self benefit from AI?

Yesterday I attended the first edition of the AITalks meetup (also here), a panel focused on AI and the impact for our digital self. The panel included the CEO of Echobox, a company who specializes in using AI tech to manage the social media profiles of companies like the Financial Times, Guardian or the New Scientist – impressive stuff, the CEO of Lobster, the “the Uber of Independent Stock Photography” (who use for example image processing to auto-tag photos or target ads) and the Director of Social Media at Motaword (couldn’t understand what the company does, but the representative was very strongly affiliated with Google).

It was in interesting night and conversation, but I have to admit I felt disappointed with what I felt were at times simplistic ways of addressing some of the topics.

On GDPR, Privacy and Personalization

One of the topics discussed was that of GDPR, and the feeling towards the regulation seemed to be of cautious resistance – to be expected from the point of view of businesses that rely on personal data to do personalization or content targeting. The CEO of Lobster said they lost 500k photos from their catalogue (if I’m not mistaken) that had been made available by theirs authors on Instagram because Facebook cut access suddenly recently and without warning, and an estimate of 60% of companies – especial smaller ones – are not ready for the regulation, a number which doesn’t surprise me.

Later in the conversation, I heard things like “I don’t see a threat in [people] sharing any data. We have to trust legal mechanisms to go after bad guys if there’s abuse” and “I think that’s a price we have to accept [the loss of privacy] in return for better and personalized services”. This worldview makes me feel extremely uncomfortable, but the CEO of Echobox did add words of caution: “If you share something on a social network, you should assume it’s now public. And you can be impersonated, someone can go and create a profile with your name, your photo, and start adding your friends. You have to be aware of that.” More a topic about privacy than AI, but without data to process there is no personalization, right? And considering the recent scandals with Facebook and Cambridge Analytica, I have to admit I didn’t like this way of seeing things.

On jobs/societal impact

Another topic I felt was addressed lightly/dismissively was the impact of AI on jobs and society, especially in the first part of the conversation before Q&A was opened.

I’ve been reading a lot of the topic, and I’m still unable to understand how can increased automation lead to MORE jobs being created by the wider adoption of AI, like Gartner is saying will happen by 2020. If 1.2 million people in the transport industry in the US alone go unemployed because of Autonomous Driving, what “meaningful and creative” jobs are they going to find? especially if AI is also starting to show up in the creative industries? I don’t see this nirvana of optimism coming to happen, and I do anticipate the need for profound Societal change especially in the “developed world”. Not tomorrow, but in time. How can a world based on Consumption be sustainable if the ones who Consume don’t have the financial means to do so? Maybe we’ll just cope and find creative ways to create meaning for ourselves, but not sure how we’ll manage the transition.
I do like the idea of “Human Augmentation” (i.e., Human + AI assisting), which was also mentioned and is easier to implement/coming sooner.

Anyway, on this topic, I think change is inevitable and it will happen (almost) inevitably, I’m hoping we’ll find ways to make it work for people and discuss it properly.

On healthcare or hiring

This was discussed more briefly, but the gist of it was that a lot of value was put in “human intuition” or “gut feeling” or “Humans having the last word”, and I personally think that both Healthcare and Hiring are precisely areas where there is more space for objectivity, especially considering the sheer quantity of cognitive biases that affect us.

Dr. House was a terrific TV Series, but he worked by trial&error – and I’d prefer not to be experimented on until suddenly (if I’m lucky) my doctor has an epiphany and finds the root cause of a problem. Or if s/he had a rough night and didn’t remember an important side-effect. And on hiring, it is known that the best predictor of future performance is past performance, and there are specific techniques that try to remove bias from interviews, such as always asking the same questions and rating candidates comparatively on them – but here I see AI’s having a harder job to help in this area (apart from ranking CVs and Linkedin profiles, maybe).

Daniel Kahneman’s “Thinking Fast and Slow” is mandatory reading for those interested in this kind of human cognitive topics.

PS

One of the replies to a question on Airplane Autopilot Systems (which reportedly have control of 97-99% of the flight time in modern commercial airplanes) was that “they aren’t AI”, which made me think of Tesler’s Theorem: ”AI is whatever hasn't been done yet.” I understood what he meant – I don’t think airplanes run neural networks, but it *IS* Artificial Intelligence. Unless, of course, flying an airplane plane doesn’t require some for of Intelligence. Winking smile.

Two years at Microsoft and a new challenge – a short reflection

Hard to believe it’s been two years since I moved to the UK and started an adventure at Microsoft!
I joined as a Cloud Solution Architect in May/2016 working with customers in Financial Services, then transitioned to Cloud Applications Solution Architect, a similar role but more focused on AppDev/PaaS services. Did the role well and got good results, and since January/2018 I eventually moved [back] to a managerial role and started as a Cloud Solution Architect Manager. I also changed industry vertical to focus now on the more dynamic Retail,Travel & Transport – where I have done more work in the totality of my career – and the new technology area of Data&AI, which actually covers a broad spectrum covering Big Data, Data Lake, Data Warehousing, Machine Learning/Data Science, cognitive services, and even IoT. It’s not an area where I have historically a lot of experience, but I’m both learning fast and  lucky to be working with an absolutely awesome team doing amazing work in some of the most important Azure customers.

One of the threads I’ve picked up recently in the Customer Success Unit is that of Artificial Intelligence, where I’m leading conversations and coordinating local investments. A fascinating area where not only Microsoft but all the big players are investing massively – Facebook and Google are names who tend to show up a lot (I’m still trying to understand what IBM Watson actually *does*, and AWS seems slightly less visible).

With Microsoft being a platform company and not focused on the delivery of services to consumers like some of the others, we tend perhaps not to be as visible or have the the same “coolness” factor, but that also frees us to focus our own investments in ethical AI and doing the right thing for our customers, under the leadership of Satya Nadella. And in the products we do deliver to consumers, like Office or Windows, you can see some of the investments in AI coming to very large numbers of people, all at once – the accessibility or translation features in PowerPoint or Skype being good examples. Together with Quantum Computing it's what interests me more in what the company is doing.

Anyway, since picking up the AI flag I’ve been posting interesting things I read on both Twitter and Linkedin (not just Microsoft contents…), so follow me if you have an interest in knowing what I’m chasing: Twitter / Linkedin.

Saturday, May 20, 2017

365 days at Microsoft UK

I completed my first year at Microsoft UK earlier this month. The fact that I haven’t been blogging is probably the best evidence for the best summary I can give: it’s been busy!

I work in a team dedicated to help some of our larger customers go to the cloud, giving guidance and architectural support across almost the entire range of Azure Services. I have retained one of my focuses from before joining Microsoft, on Azure Service Fabric, and have effectively become the go-to person in the team for this technology.

Working for a large American company – in the largest subsidiary after the US - with a deep hierarchy, moving from a small and agile company, was a big change. One thing I have found curious is that I have more autonomy and freedom in the way I approach each customer engagement or manage my time and where I work than I ever had at my own company. The culture of autonomy and collaboration runs deep, with everybody being encouraged to grow, rely and help others, and the fact that I’m working in what is by far the most interesting area of what the company is doing – Microsoft Azure – is a large part of what makes the challenge interesting. I have to admit I miss some things: working in close knit teams, having more technical depth with concrete problems, and having more participation in decisions that affect me personally.

I not only changed company, but also countries. I quickly noticed some differences between Portugal and the UK:

  • I have spent more time on the phone and conference calls in this last year than in the entirety of my career! The headset is always with me. Most of the meetings, either with colleagues or customers, happen over Skype. Even if f2f is important, traveling is expensive and time consuming.
  • The meetings start on time and very rarely overrun;
  • No “solutionizing” in meetings – most of the meetings I attend are either to communicate or to make decisions. Work is done offline. Most of the meetings, either with colleagues or customers, happen over Skype. Even if f2f is important, traveling is expensive and time consuming.
  • A lot can be achieved with 20 or 30 minute calls, if people are disciplined and focused. This was unheard of, back in Portugal.
  • When I came, my lunch slot was almost always hijacked by meetings. Days where I have 6 calls back to back are common, and if I want to protect my time (and sanity!) I have refuse meetings at lunch time, and make sure I reserve time for things like travel [to London or other places] or to do self-training.
  • A lot of work is done remotely, and the company doesn’t really care where I am physically as long as I am working towards reaching my targets and goals.

Regarding being in the UK in an extra professional environment, I do feel a big difference in scale and challenge, but not everything is positive: Brexit affected me (personally) a lot and hurt my image of the country massively – even if London is a world apart. Also, this country has a serious problem with infrastructure, from telecoms (e.g., GSM or 3G coverage) to public transportation with constant delays, problems and strikes. Comparatively, in Portugal there is a a service to be proud of! 

I had been lacking the inspiration and time to look back at this past year, but today I had by evaluation meeting, and thought it appropriate to retrospect. So there.

Saturday, May 14, 2016

Joined Microsoft UK

Great news! After several years working closely with people at Microsoft in Lisbon as a CTO of Create It, as of Monday May 9, I have joined Microsoft in the UK subsidiary. I'll be working as a Cloud Solutions Architect in the EPG team, completely focused on Azure, my topic of choice in the last few years.

This blog will continue existing, hopefully with more frequent posts and how more focused on Azure Architecture. One disclaimer I must do is that the opinions and views expressed in this blog are mine and do not necessarily state or reflect those of Microsoft. Just so you know :-).

Wish me luck!

Tuesday, February 23, 2016

How (not) to hire - notes on hack.summit()’s session

Pluralsight’s virtual hack.summit() event started yesterday, and I managed to follow most of the sessions. The last session was by Hampton Catlin (“Created Sass, Haml, m.wikipedia.org, and book author") on the topic of “How (not) to Hire”. The session is partially a reflection on the success of past hires, but also the sharing of alternative and (lots of) interesting ideas on how to hire people/developers.

One of ideas I thought was most relevant was that you WILL do bad hires, and that you should face the idea of letting people go as naturally as possible: “It’s your job as a manager”. I’ve had to do it in the past, and it’s not easy, but he’s right. “What are you afraid of when hiring someone? that he’ll bring down the company?” These questions frame the process in a “lighter way”: you don’t have to get it right at first, and should accept that you'll make mistakes. I guess most of us are so cautious when hiring because of the effort/time associated, but also because of the failure itself and being associated with it. I’ve both done bad hires in the past, and also seen the “stigma” associated with it in a colleague, so accepting that mistakes will inevitably happen should be in our minds to avoid this - no mistakes, no learning.

Another relevant insight from the session was the difference between Positive and Negative hiring. In Negative hiring, you try to find problems, reasons NOT to hire. In Positive hiring, you try to find out how awesome someone can be, what are his strong points, how he can make a difference. I bet most of us that hire follow the first approach (me included). He says that he used to follow the “Veto Rule”, whereby if anyone on the team wasn’t convinced the interviewee was awesome, they wouldn’t hire. This is exactly what I did for most of my career, on the footsteps of “Peopleware”’s recommendation to involve the team in the hiring process. An alternative approach, he mentions, is the “Believer Rule”, whereby if anyone on the team is really convinced someone would be awesome, he should be hired. The truth is probably somewhere in the middle, but this a great idea.

Catlin also advocates that you should hire mostly depending on someone's ability to work as a team/collaborate, and not only technical skills – “It’s easier to teach code than it is to teach empathy”. This is true, but I’d say there must also be a technical baseline.

Another idea, and this was to me the most interesting one, was the suggestion that candidates themselves should design their own hiring process, giving them a chance to show off and see their best side. What the candidate chooses is obviously a clue into what his strong and weak points are. I find this idea brilliant, and will keep this in mind for the future.

Another suggestion that he gives is that just discussing technology with candidates is a very effective way of assessing the person, and his/her passion, interest and skill in technology. Asking about some specific project, what they think of Azure vs AWS, or of C# vs Python, etc.

Finally, during the Q&A there was another interesting insight, or framing of the hiring process overall. Some companies (I think I’ve read about Google doing this) try to hire people that are better than the average of the company, constantly “raising the bar”. The insight is this: if you keep doing this, at some moment in time you yourself should NOT be employed at the company! :)

Anyway, lots of good ideas in the session, to keep in mind for the future.

Thursday, February 11, 2016

SIGFOX Makers tour Lisbon

I just spent the afternoon at SIGFOX’s event in Lisbon, learning about the company offering and doing some hands-on demos with an Arduino Uno and SIGFOX’s network.
What is SIGFOX? To be honest, I didn’t exactly know, when I got the Meetup email from Productized about the event. SIGFOX is an telecom operator, exploring a range of radio frequencies and a kind of modulation (UNB – Ultra-Narrow Band) that makes it especially interesting for the IoT world.

Up until today, my only experience with IoT was with the Raspberry Pi, and the code to handle the wifi connectivity in case of failures has given me several headaches. It seems bluetooth, which is more widely used with Arduinos, is also not a simple process, due to the need to pair, for example. SIGFOX’s network promises several advantages in this area: simplicity in sending (and receving messages) from the network is a question of doing a send call – no need to pair, connect, authenticate, etc. There are base stations (much like in GSM) that get the message, de-duplicate it, make it reach SIGFOX’s infrastructure, and either process it there or route it to our own application server via an HTTP call. Another major selling point is the low power usage: the protocol used is designed to maximize energy efficiency and use energy only when needed - essentially, it uses 20-35mA when sending messages, and the chip is supposed to be in standby most of the time, when not communicating – the communications pattern assumes the device polls for incoming commands when required/regularly. A solution using this approach is supposed to have up to have several years of autonomy, supposedly.

This simplicity and low power consumption, clearly major selling points, are offset by some limitations in the range of applications: messages have a payload of 12 bytes (xml and json are out, as are applications regarding media transmission) – this means you’ll be optimizing messages on the bit level; additionally, to comply with european regulations in the usage of the spectrum, you can only send up to 140 messages per day (about one message every 10 minutes), and finally the transmission rate is pretty low: 100 bits/sec – so it pays to keep messages as small as possible. Even with these limitations, it’s pretty clear there is a wide range of applications for this approach – my Raspberry Pi temperature setup uses Wifi and is constantly plugged in, a SIGFOX+Arduino solution would probably just require batteries.

Interestingly, there is already network coverage in Portugal (an in the Netherlands, France and Spain – as well as several other major cities worldwide), and prices are supposed to be very affordable (2€ to 25€/year, if I understood correctly), depending on factors such as usage. In one of the demos, a message sent by the board was picked up by 4 base stations – the workshop was in the city’s downtown.
Snootlab_Akero_ArduinoUno
The platform has mechanisms (callbacks) that can handle requests from the devices, either semi-automatically by parametrizing preset responses (ex: return the current time), or by making HTTP calls to a configured application server on our end. Interestingly, Azure Event Hubs (which I also use in the Raspberry Pi tests) are also natively supported, with the network automatically posting incoming messages from devices into a configured event hub – and the Azure IoT Suite will also be supported.

I thoroughly enjoyed the afternoon in the workshop, and was surprised at how easy it was to use the network and the Arduino. I’d seen demos of Arduino before, and to be honest was not looking forward to be back in coding C, but it was easier than it thought (I still have this book around, from when I learned the language in high school).

As to doubts and issues: IoT security seems to be in the news every day, so this is obviously a concern. I don’t know much on the topic, but device authentication, server authorization, and server authentication in the device seem obvious concerns. The trainer, Nicolas (great job!), didn’t have much time so he didn’t expand on the topic (and I still don’t have the slides with me), but it is a concern I’ll have to explore.

The SIGFOX demos done in the workshop are available on GitHub. The board provided can read the temperature (with a degree resolution), so I might do the exercise of converting my C# code from the Raspberry to run on the Arduino.

PS: I have very basic electricity/electronics know-how, and somehow see myself with a couple of Raspberry’s, now an Arduino, and several sensors, cables and breadboards on the way from AliExpress. What is happening to me? :-)

Monday, January 11, 2016

IoT: Raspberry Pi2 and Wireless conectivity losses

As I have described elsewhere, I have setup my Pi2 with a temperature sensor which posts the information to an Azure Event Hub. I’ve made the code more robust to handle different errors, but eventually the Pi2 – every 3 to 4 days - would just stop sending readings and responding to pings.
The problem is apparently related to the Wifi dongle, so (with a little help from a friend) I found the commands that restart the wlan, and after adding the code, the problem seems to be fixed. Here’s the relevant C# code:

public static void RestartWifi()
{
    Process proc = new Process
    {
        StartInfo = new ProcessStartInfo
        {
            FileName = "bash",
            Arguments = "-c '" + @"/sbin/ifdown \'wlan0\'" + "'",
            UseShellExecute = false,
            RedirectStandardOutput = true,
            CreateNoWindow = true
        }
    };

    proc.Start();
    Thread.Sleep(5000);
    proc = new Process
    {
        StartInfo = new ProcessStartInfo
        {
            FileName = "bash",
            Arguments = "-c '" + @"/sbin/ifup --force \'wlan0\'" + "'",
            UseShellExecute = false,
            RedirectStandardOutput = true,
            CreateNoWindow = true
        }
    };

    proc.Start();
}

Thursday, January 7, 2016

Videos of 2015’s sessions online

Just found out about 3 videos of recent sessions being published online, so I’m posting the links.
The video of my WebCamp 2015 session on Azure WebApps is now online at Channel 9. The 45 minutes were clearly not enough for everything I wanted to show! My summary of the session is here.

Also available at Channel 9 is the session I did earlier this year at TechRefresh 2015 on the Azure App Service architecture, where I focused much more on Azure LogicApps. My summary of the session is here.

Finally, the video stream of the roundtable at Merge Lisbon #3 is available now on Youtube. The roundtable itself starts at around 01h45m. My summary of the session is here.

Monday, January 4, 2016

«Creating a great [software] engineering culture» roundtable at Merge Lisbon #3 – Post event thoughts

(this post is somewhat late, but this time of the year is always busy).

I thoroughly enjoyed the roundtable at Merge Lisbon #3. It was a very interesting and lively discussion, and having two people coming from the online product world/startup, plus two from the development/system integration services world, contributed a lot to the discussion and exchange of different points of view. RUPEAL’s organization was top-notch, and I’ll be waiting for the future editions of Merge Lisbon.

Like I said, there were several topics on the table, starting with what creates a great engineering culture, what is culture anyway, and what is the influence of the “founding fathers” of a company in its culture (”if the founders live 10.000km away, not much”, was one of the answers).
Having founded a services company, of which I was the CTO for 14 years, these are questions that made me think and that I feel as my own. In my view, my company was a mix of the personal views of each of the founders (which mostly – but obviousy not exacly – coincided). A vision of values like agility, transparency, qualitity in the delivery, informality, etc. In time, I do think that this changed - one day at a time - towards a more processual way of working, a trend that was luckily identified and reversed.

Reflecting on my own experience, I think that a founder working in his own company has a bond that is much stronger than that of employees, who can leave whenever they want. Making sure that he creates an adequate workspace, with a great team, challenging work, great culture – is essential to his own happyness and the company itself. And this leads me to one of the points mentioned that night, that a lot of this culture issue relates to “fit” - between the people and the founders and the way the company does business. Which reminds me of the recent example of Holacracy implementation at Zappos – and if I can empathise with the wish to have a specific way of managing the company, I can’t feel the same regarding the “either you’re in, or leave” way of putting things. I’m sure Zappos lost amazing talent in that process. Was it worth it?

One of the big topics discussed at the roundtable was the Hiring process, and how to decide who to hire. My take is that while technical skills are critical, behavioural fit is also as essential, and being able to work as a team. One reference I gave out was the (absolutelly mandatory) book Peopleware by Tom DeMarco and Timothy Lister, which suggests that the team which the new employee is joining should be involved in the hiring/interview process. Other books that have shaped my view on this are Belbin’s Management Teams: Why they work or fail and Team Roles at Work. These defend that – at least in management teams – complementarity between the “types” of people is what makes the teams work. One extreme example is: does anyone believe a team of 5 people like Steve Jobs would get anything done?

Some roles Belbin identifies are the Shaper (the classical “I Have a Vision” guy), the Chairman/Coordinator, the Creative (“Plant”), etc. Joining these views together leads, however, to a very real risk: that of creating a “mono culture”, where people who have very different team roles don’t get hired. For example, a company with a focus on “conservative” team roles will tend not to hire creatives/disruptors (“Plant” and “Resource Investigator”, in Belbin’s terms). Would you hire someone like Elon Musk for your team, if you were recruting him?
I do ask myself if this role categorization captures all there is to it, and tend to think it doesn’t. In my first job, over 90% of the people (web designers/creatives, all) voted in the same left-wing party, for example. Some more food for thought.

A specific question from the audience was on the value of having handbooks with the “principles” and vision of a company, much like Valve’s Handbook for new Employees. While I agree that these documents can be seen as a way of “marketing”, I still see value in them. They represent a vision for what the company strives to be, helping keeping focus/remembering of what the core values are. Time generaly turns organizations into the convervative/bureocrat side (as Jullian Birkinshaw brilliant argued in his Coursera course on Managing the Company of the Future), and these documents can help fight that tendency. Other examples I know of are Ricardo Semler’s “Employee Survival Manual” at Semco (a sample of which is at the end of his book, Maverick), and recently I found out about Outsystem’s The Small Book of The Few Big Rules as a a great portuguese example.

The topic of being a specialist (someone with specific skills in a given area of technology) or a generalist (broad know-howin several areas of technology) was also on the table. The trend seemed to tend towards a tendency to prefer specialists in the hiring process, people who can quickly deliver results in their area of expertise. My only take here was a sentence from an experienced software architect I admire professionally, when describing himself: “I’m an expert in generalities”. This is somewhat obvious if you think about it, but as you grow in experience you will start doing architecture, and it’ll be impossible to know all the technical details of the different parts of your solutions. You’ll have to focus more on knowing the capabilities, and forget about knowing the specific details of each technology. In my experience, making this leap (realizing you don’t have to be an expert in everything) is perhaps the most important one on the career path from developer to tech lead/architect.

Another of the questions from the audience was on how to keep the culture of a company when a team is working at a customer’s site. This is a challenge I am well aware of, because of my work in the last 4 years, and my answer to this was to take your company’s culture with you. One example (a small thing) is just shake everybody’s hands when reaching the site – incluing the client’s people - , and going for a cup of coffee together [as you would in your office]. But this is a simplistic answer and example. Other factors that I think are important are: try to bring people over to your office regularly, even if just for half a day every two weeks, have leadership/management people work at the site as well (even if on different contexts), in internal communication try not to forget that some people are out of the office (for example, if scheduling training sessions, birthday lunches, etc.). “But what if the client’s culture turns out to be better than yours?” “That’s good on two ways: first, you’ve learned something about what motivates your team, and second, you can try to bring that experience to your own company”.

There is a promotional video of the event, which you can checkout at KWAN’s Facebook page, for a small taste of what it was.

Thursday, December 10, 2015

«Creating a great [software] engineering culture» roundtable at Merge Lisbon #3

This evening I’ll be participating, together with people from local companies Uniplaces, Talkdesk and GrandUnion, in a panel on how to create a great software engineering culture, at the third meeting of Merge Lisbon.

An interesting topic, in a world were startups are born everyday focused on delivering an MVP as soon as possible (and not quality), where people stay less and less time in their jobs or simply freelance and pick only the really enticing projects, where Agile can on occasion lead to sloppiness, where people (rightly) enjoy working from home, and where – very often – the “bits are worth less”, for example when developing microsites for events or apps for short-lived festivals. Being based in Lisbon, I could also add: and where people leave all the time to work in the UK, the Netherlands or Switzerland.

I’ll come back here after the panel, surely with new ideas.

More information here (but it’s been fully booked for some time, I’m told).

Monday, December 7, 2015

[Micro-post] Puretext has a new version!

I’ve been using this little tool for years now, and just noticed it has a new version. PureText is a simple tool that stays in your tray icon, uses up 0,1Mb in RAM (!), and allows you to do Windows-V to paste text, removing formatting. You can change the key combination, but this one is perfect.

Extremely useful! Completely free, downloadable here.

Friday, December 4, 2015

«Azure WebApps: Why aren’t you using them yet?» @ Microsoft WebCamp 2015 - Lisboa

Microsoft Portugal held the yearly WebCamp event this last Wednesday, focused on Web Technologies,both from Microsoft and Open Source. The openness to OSS is clear, with sessions focusing specific to that approach to software development. The term “best of breed” does come to mind, in the sense that more and more solutions include parts by different sources/vendors, that have to work well together.

Anyway, my session was focused on Azure WebApps, and I expected the technology to be familiar to most people by now, but was surprised to find out it is not. The session had an enterprise-ready focus, which I had to adjust somewhat as a consequence.

As usual, the session was demo-heavy, and unfortunately the 40 minutes were not enough to show them all. Here’s what I had planned:

  • Create a site and publish from Visual Studio – showing the simplicity of the process and the new window in the SDK 2.8.1;
  • WebApps in the Azure Resource Manager – open up https://resources.azure.com and show how the site resources and their settings are represented;
  • Backup and Restore – using a site (a to-do app sample) I had previously deployed, I showed how the Backup and Restore features work, including the database-embed capability. For me, one of the killer features of this PaaS offering;
  • Remote Debugging – it’s always amazing, being able to do remote debug to code that is running remotely in the cloud. There was a specific session on Application Insights, so I opted to not go into it;
  • Staged Publishing – deployment slots are huge, both in terms of supporting the dev/test/quality environments, but also in team development itself (e.g., having a slot for each developer). Had a chance to show a swap and the “setting stickyness”, but had to skip the “A/B testing” support (where x% of the traffic is directed to a slot, and the rest to another one);
  • Traffic Manager – this is another impactful demo: having pre-created sites in North Europe, Brazil and Japan, used traffic manager to unify them under a single domain name, and then www.whatsmydns.net to check the dynamic resolution. Always a great demo (I’m impressed myself Smile);

The session time was only 40 minutes, so other demos had to be left out:

  • Redis Session State Provider – the idea here was to explain about Redis (an instance of which I had pre-created), install the package with the session state provider, change web.config, and redis-cli to show the keys/sessions being added to the repository;
  • Scaling – having a Redis has the backend to the user’s session state, the obvious next demo was using Scaling/Automatic scaling to see it working, and showing how scaling back down to one single server didn’t imply a loss of session;
  • Web Tests (not to be confused with Load Tests) – one of the newest features in WebApps is Web Tests, a service that works similarly to what services like AreMySitesUp provide: access your site from several locations worldwide, and if for example 3 of them can’t reach it more than 3 times, send and email alert. Discrete, but helpful.
  • IP Blocking – this final demo was addressing one specific complaint about the way IP Blocking works in Azure WebApps: either you include the blocked IPs in Web.config, or use App Service Environment (ASE). ASE implies the Premium service tier, which is costly. Adding an IP in the web.config implies a site restart, and if you have sites in several regions, you have to make the change in all of them. So the demo goes the applicational path: the IPs to block are simply added to Redis (ex: IPB1.2.3.4) and an Action Filter in the MVC project checks the source IP and returns HTTP 403’s if it’s in the blocked list. Quick and yet not dirty Smile.

That was it. A full room, I’m just sorry I didn’t have the time to show everything I had prepared. Maybe I should do some screencasts?

Monday, November 16, 2015

Revista Programar: Azure Logic Apps: o futuro dos backends? [Portuguese]

My article "Azure Logic Apps: o futuro dos backends?" just made the cover of the 50th edition of the "Revista Programar" magazine. The article describes my view of the historical evolution from Mashups to SOA to Microservices, and describes the current version of Azure Logic Apps, Microsoft's implementation of that architectural view.

If you do happen to read portuguese :), the direct link to the article is here.