Table 5 Questions for surveys S₁, S₂ and S₃

S₁

Q1

-

-

Please give an overall rating to the challenge

Q2

-

-

How well could you recognize the vulnerability in the code?

Q3

-

-

How well can you fix this problem in production code?

S₂

F1

(Graziotin et al. 2018)

HP

My overall experience with the platform was positive

F2

(Hänsch and Benenson 2014)

BE

The Sifu platform helps me to improve my secure coding skills

F3

(Hänsch and Benenson 2014)

PE

Solving challenges in the Sifu platform helps me in recognizing

vulnerable code

F4

(Hänsch and Benenson 2014)

PR

Solving challenges in the Sifu platform helps me in understanding

consequences of exploiting vulnerable code

F5

(Graziotin et al. 2018)

HP

Solving challenges in the Sifu platform makes me overall happy

F6

(Hänsch and Benenson 2014)

BE

Challenges in the Sifu platform help me to practice secure coding

guidelines

F7

(Graziotin et al. 2018)

HP

I find the Sifu platform adequate as a means to raise awareness

on secure coding

F8

(Graziotin et al. 2018)

HP

The examples in the Sifu platform are clearly presented

F9

(Graziotin et al. 2018)

HP

It is fun to solve challenges in the Sifu platform

S₃

X1

(Hänsch and Benenson 2014)

PE

I have learned new techniques and principles related to secure

software development

X2

(Hänsch and Benenson 2014)

PR

The challenges help me to understand possible consequences of

security breaches

X3

(Hänsch and Benenson 2014)

BE

I feel I am better prepared to handle secure coding-related issues

at work

X4

(Hänsch and Benenson 2014)

BE

The challenges help me to understand the need to have a well

defined secure software development lifecycle activities

X5

(Hänsch and Benenson 2014)

PE

I’ve learned about new issues related to security that can occur

during software development through playing the challenges

X6

(Graziotin et al. 2018)

HP

The help from the coaches was adequate

X7

-

WK

I feel more prepared to understand the output of static

application security testing tools

X8

-

WK

Through playing the challenges, I know where I can find

information about secure coding guidelines

X9

(Hänsch and Benenson 2014)

PR

Through playing the challenges I understand the importance of

using secure coding guidelines

X10

(Hänsch and Benenson 2014)

BE

Through playing the challenges I feel my practical secure coding

skills have improved

X11

(Graziotin et al. 2018)

HP

The challenges are related with my daily work in my company