/ Cloud

Happiness Monitoring In The Cloud

Do you think that monitoring and reporting your company happiness is impossible? In this post, I'll describe Quantyca Happiness monitoring, a serverless solution to deliver happiness surveys to our colleagues.

From the problem…

We can monitor everything, except for our team morale. We decided to fix it.

What is corporate happiness? Have you ever thought about asking your employees this question? We do!

We spend a lot of our time in the office. It is an ecosystem where we interact with different people, complex situations (projects, technologies) in different environments (offices, customer site, etc.). So we started a project with the goal of measuring our corporate happiness!

We work on the concept of aggregate work happiness. Day by day, week by week we will study it and try to understand its dynamics, the elements, and events that contribute to its evolution. We will observe aggregate trends and also personal ones.

Happiness index will inspire moments of reflection and action both on the individual person and on the group.

...to the solution


What do we ask to our colleagues? To vote.

Express weekly their sentiment about the week just passed, about relationship with colleagues or customers, progress of the projects, technological achievements and acquisition of new skills, recognition of our work, problem solving, etc..

The idea is simple:

  • just asking our teammates how it is going
  • gather happiness indexes in a table
  • and then build an evergreen old-school report that can show us trends

So simple that we had to come up with something to complicate it.

The Architecture


First of all, we wanted to have some fun with Google Cloud Platform.  To do so, we took a serverless approach, integrated with the already existing DBMS. Thus we built from scratch an infrastructure to send email based surveys and gather happiness indexes.

The following image shows an overview of the architecture we built:

The components of the infrastructure can be separated according to three different concerns:

  • Campaign Management: generates the campaign by selecting the target email addresses. It defines the campaign parameters and eventually sends the customized email.
  • Happiness Index Gathering: a micro-application allows the user to send the survey results by clicking on a bunch of smilies in the email.
  • Reporting: the final report where all the obscure mysteries are revealed. It is backed by our company HR  DBMS.

To keep all this stuff secure we looked for a cheap and completely stateless solution. So, we opted for the use of JSON Web Tokens. JWTs are generated by the campaign manager and filled with the correct parameters. In this way, only a user who received an original email from the campaign manager can submit his/her weekly happiness index. Nonetheless, the certified parameters in the JWT prevent jokes from colleagues changing each other results.

The Happiness Journey

Campaign manager: target list extraction

Everything starts with Cloud Scheduler. Every Friday at 1 PM it publishes a message on a PubSub topic that almost immediately triggers the first Cloud Function.

Its duty is to extract from our DBMS the target list of the campaign. For each individual in the target list, a new message is published on another topic. This message contains the user id, the email address, and the campaign date: all the parameters needed to send the email and gather the data correctly.

Campaign manager: email sending

This second message wakes up the second cloud function in charge of:

  • generating the JWT containing user identifiers and campaign date
  • populating the email template
  • eventually sending the email
let jwt = require('jsonwebtoken');
let nodemailer = require('nodemailer');
let config = {
  TOKEN_EXPIRATION: process.env.TOKEN_EXPIRATION || 60 * 60 * 24 * 5,
  JWT_SECRET: process.env.JWT_SECRET || 'change_me!',
  MAIL_HOST: process.env.MAIL_HOST || 'smtp_not_provided',
  MAIL_PORT: process.env.MAIL_PORT || 587,
  MAIL_USER: process.env.MAIL_USER,
  MAIL_PASSWORD: process.env.MAIL_PASSWORD,
  MAIL_SECURE: process.env.MAIL_SECURE || 'false',
  MAIL_FROM_ADDRESS:
    process.env.MAIL_FROM_ADDRESS || 'happiness@yourdomain.com'
};

/**
 * Triggered from a message on a Cloud Pub/Sub topic.
 *
 * @param {!Object} event Event payload.
 * @param {!Object} context Metadata for the event.
 */
exports.onSendEmail = async (event, context) => {
  const pubsubMessage = event.data;
  const payloadString = Buffer.from(pubsubMessage, 'base64').toString();
  console.log(payloadString);
  const payload = JSON.parse(payloadString);
  let { email, date, userId } = payload;
  let token = generateJwtToken(email, date, userId);
  return await sendEmail(email, date, token);
};

function generateJwtToken(email, date, userId) {
  let token = jwt.sign(
    {
      date,
      userId
    },
    config.JWT_SECRET,
    {
      subject: email,
      expiresIn: config.TOKEN_EXPIRATION //seconds
    }
  );
  return token;
}

function sendEmail(email, date, token) {
  let mailOptions = {
    from: config.MAIL_FROM_ADDRESS, 
    to: email,
    subject: `Quantyca Happiness ${date}`,
    text: 'This email is in html only',
    html: emailTemplate.replace(/replacetoken/g, token)
  };
  let transporter = nodemailer.createTransport({
    host: config.MAIL_HOST,
    port: config.MAIL_PORT,
    secure: config.MAIL_SECURE == 'true',
    auth: {
      user: config.MAIL_USER,
      pass: config.MAIL_PASSWORD
    }
  });
  return transporter.sendMail(mailOptions);
}

const emailTemplate = "...email template...";

Gathering data

Eventually, a notification pops up on the team smartphones. A new incoming email!

The click of a smiley triggers the third function and the happiness index is written in a table of the Cloud SQL DB.

const mysql = require('mysql');
const jwt = require('jsonwebtoken');

/**
 * specify SQL connection details
 */
const connectionName =
  process.env.INSTANCE_CONNECTION_NAME || '<YOUR INSTANCE CONNECTION NAME>';
const dbUser = process.env.SQL_USER || '<YOUR DB USER>';
const dbPassword = process.env.SQL_PASSWORD || '<YOUR DB PASSWORD>';
const dbName = process.env.SQL_NAME || '<YOUR DB NAME>';
const JWT_SECRET = process.env.JWT_SECRET || '<YOUR JWT SECRET>';

const mysqlConfig = {
  connectionLimit: 1,
  user: dbUser,
  password: dbPassword,
  database: dbName
};
if (process.env.NODE_ENV === 'production') {
  mysqlConfig.socketPath = `/cloudsql/${connectionName}`;
}

// Connection pools reuse connections between invocations,
// and handle dropped or expired connections automatically.
let mysqlPool;

/**
 * Responds to any HTTP request.
 *
 * @param {!express:Request} req HTTP request context.
 * @param {!express:Response} res HTTP response context.
 */
function entry(req, res) {
  jwtVerify(req, res, routing);
}

function jwtVerify(req, res, next) {
  let token = req.query.token;
  try {
    jwt.verify(token, JWT_SECRET);
    next(req, res);
  } catch (err) {
    console.log(err);
    res.sendStatus(401);
  }
}

function routing(req, res) {
  switch (req.method) {
    case 'GET':
      insertHappinessIndex(req, res);
      break;
    default:
      res.sendStatus(405);
  }
}

function insertHappinessIndex(req, res) {
  // Initialize the pool lazily, in case SQL access isn't needed for this
  // GCF instance. Doing so minimizes the number of active SQL connections,
  // which helps keep your GCF instances under SQL connection limits.
  if (!mysqlPool) {
    mysqlPool = mysql.createPool(mysqlConfig);
  }
  let { userId, date } = jwt.decode(req.query.token);
  let happinessIndex = req.query.happinessIndex;

  mysqlPool.query(
    {
      sql:
        `insert into qtrack.HAPPINESS_INDEX (USER_ID,DATE_ID,HAPPINESS_INDEX) 
         values (?,?,?) on duplicate key update HAPPINESS_INDEX = VALUES(HAPPINESS_INDEX)`,
      values: [userId, date, happinessIndex]
    },
    (err, results) => {
      if (err) {
        console.error(err);
        res.sendStatus(500);
      } else {
        res.send(responseHtml);
      }
    }
  );

  // Close any SQL resources that were declared inside this function.
  // Keep any declared in global scope (e.g. mysqlPool) for later reuse.
}

const responseHtml = `
<!DOCTYPE HTML>
<html>
	<head>
		<title>Quantyca Happiness</title>
	</head>
	<body style="margin: 0; background-color: #f1f1f1;">
    ...
	</body>
</html>
`;

exports.entry = entry;

Reporting

The last step of this journey is to finally see the data through a Data Studio report. Here it is!

Conclusion

  • Time spent: low
  • Infrastructure Costs: free tier
  • Fun & happiness of the developers: high

Many thanks to Matteo, Alessia and Giandomenico that participated with happiness in this little crazy project.