Goal Experiment Summary Report: Survey Pop

After the first attempt at setting up a goal experiment on my Angular app didn’t work, I conducted research on how to correct this issue.

I found an Angular service called angular-google-analytics which provides a way to use Google Analytics on Angular Apps.

It can be installed using Bower and there are a lot of configuration options that you can put into the App.js file including event tracking.

When I tried adding the service to my application I was experiencing errors so I decided to go to Plan B which is running an experiment on my User Journal.

I created two variations of the article “Content Experiment on Survey Pop.” The original version has a text link to the Survey Pop application and the variation has a green button that links to the Survey Pop application.

content-variations

What were the results of your Content Experiment? Did they match what you thought would happen?

After four days of data collection, there were some preliminary results available.

experiment-data

So far, the variation is outperforming the original with 2 conversions in 4 experiment sessions. The original also had 2 conversions but in 6 experiment sessions. The conversion rate for the variation is 50%, versus 33% for the original.

CNN can now project that variation 1 will be the winner of the experiment. Just kidding. Google Analytics gives variation 1 a 68.9% probability of outperforming the original.

My hypothesis is that the green button is helping to encourage people to click on the link because it stands out against the white background of the page. I predicted that the green button would increase conversions, and so far the data supports this prediction.

Have you been tempted to use less ethical means of converting users? (e.g. Misrepresenting links, spamming, false keywords, etc.)

I have not been tempted to mislead the visitor to increase conversions. This seems counterproductive because if the user doesn’t convert by their own accord, they probably won’t be willing to make a purchase or anything else that requires some effort.

This question reminds me of the tactic of popup windows to ask the user for an email newsletter signup. This is controversial because it can be annoying to the user, but there is probably good data that this tactic increases conversions significantly. If I decided to use this tactic, I might delay the popup so only people who have spent some time on the website would see the popup, instead of getting a popup right when they first visit the website.

Are these tools (Google Analytics) usable? Have they been challenging for you?
It was great learning about how to implement content experiments on a website. It seems pretty simple to implement on a site although I was not able to get the experiment to work on an Angular application because Google thought that there were only one or two pages on the site instead of the actual page count (6-8 pages). I wasn’t able to get the google analytics service to work on my app without errors. I think it would help to use a service like Prerender.io which creates a static version of each page and makes Angular apps more “Google friendly.”

Content Experiment on Survey Pop

I set up a A/B test on the homepage of Survey Pop to test whether adding a button increased visits to the survey page.

First I created a new angular route with Yeoman for my angular app called mainvariation and copied the homepage HTML to mainvariation.html. There is a slight change between the two versions which is adding a button to the link “View the Survey”.

home-page-variations

I then set up a new experiment in Google Analytics which included adding a goal of a visit to the survey page. I set the goal of visiting the survey page as the criteria for a successful outcome. I then added the JavaScript code to the head section of my Index.html.

Google Analytics confirmed that the code was working on the page I am testing. I solicited test subjects at the course Slack channel and GA tracked 10 sessions in the first day.

validate-code

After 6 days I rechecked the results of the experiment and there were still zero goal conversions for both variations.

experiment-results

After some investigation, I found that Google Analytics thinks that my app only has two pages.

content-drilldown

Since, Google Analytics can not tell that the survey page is a different page than the home page of the app, it can not determine when a goal has been completed. I haven’t found a workaround for this issue yet.

Content Experiment on Survey Pop

I set up a A/B test on the homepage of Survey Pop to test whether adding a button increased visits to the survey page.

First I created a new angular route with Yeoman for my angular app called mainvariation and copied the homepage HTML to mainvariation.html. There is a slight change between the two versions which is adding a button to the link “View the Survey”.

home-page-variations

I then set up a new experiment in Google Analytics which included adding a goal of a visit to the survey page. I set the goal of visiting the survey page as the criteria for a successful outcome. I then added the JavaScript code to the head section of my Index.html.

Google Analytics confirmed that the code was working on the page I am testing. I solicited test subjects at the course Slack channel and GA tracked 10 sessions in the first day.

validate-code

After 6 days I rechecked the results of the experiment and there were still zero goal conversions for both variations.

experiment-results

After some investigation, I found that Google Analytics thinks that my app only has two pages.

content-drilldown

Since, Google Analytics can not tell that the survey page is a different page than the home page of the app, it can not determine when a goal has been completed. I haven’t found a workaround for this issue yet.

Adding Goal Tracking: CharlesSipe.com

I am adding goal tracking to my personal blog CharlesSipe.com.

charles-sipe-website

I would like to track the goal of clicking on a link to view my current resume but that might not produce enough data for the purposes of this assignment. Instead I will track the goal of spending more than three minutes on my blog. This will tell me whether people are immediately bouncing or spending some time reading my blog.

In July my blog had 580 users visit the site with an average session time of just 54 seconds.

I added a goal of spending at least 3 minutes on the site and set an arbitrary $1 value for experimental purposes. The test showed that there was a 8.33% conversion rate in the last 7 days.

goal-setup

goal-setup-2

Easy peasy.

After a couple days I checked the status of the goal tracking.

I had 6 goal conversions on August 12th but 0 goal conversions on August 13th. It also shows the page where the conversion took place in the conversion report page.

Analytics Report: Ranked Blogs

For this project, I am reviewing the Google Analytics of a website project I built with the help of a mentor last year at Rankedblogs.com. It is an Angular application with the purpose of ranking the best blogs in different categories based on votes and website data from the Moz API.

My predictions about the traffic:

My site is most popular with women, ages 25-45, from the US.
Most of my users come from links from blogs.
The most popular search terms that lead people to my site are “ranked blogs”, “dad blogs”, and “code school blogs”.
The most popular page on my site are my top dad blogs.

Findings:

A majority of traffic is from the United States with 137 sessions (67.49%) in the past month. The second largest country is the United Kingdom with 26 sessions (12.81%) in the past month.

Most users are currently coming from direct traffic (89 out of 203 sessions). Referrals was the second largest source of traffic (50 sessions) followed by organic search (38 sessions) and social (26 sessions).

The most popular search terms were not available. 37 out of the 38 sessions from search were “not provided.” The one session with data had the search term of “top mom blogs 2016.”

The most popular page is the craft blogs with 100 sessions in the last month (29.41% of sessions). This was followed by mom blogs with 57 sessions (16.76%), dad blogs with 38 sessions (11.18%), and farming blogs also with 38 sessions (11.18%).

The average time on site was 3 minutes and 55 seconds while the bounce rate was 79.31% and the exit rate was 59.71%.

I learned that you have to enable demographic reports in your Analytics account to view the demographic data from your site.

demographic-reports

I also enabled search console to get additional data like the devices visitors were using.

search-console

Adding Transcripts to a YouTube Video

I uploaded a presentation I recorded in the first quarter of the Seattle University Web Development Certificate titled “The WordPress Switch to JavaScript.”

I went into the closed captioning settings to view the transcript of the video where I noticed there were a lot of errors and YouTube does not know how to put periods or question marks at the end of sentences.

There were more errors than I expected and it did take quite a bit of work to correct all the errors and add all the periods. It doesn’t seem to understand brand names very well such as Angular or WordPress so you usually have to fix those errors. Some of the errors are hilarious like mistaking “pull requests” with “poetry class.”

I am not too surprised that there were so many errors. On one hand I thought that Google would have better transcription technology because they do transcription a lot with Android’s voice commands. However, it is a difficult problem for computers to understand language and I’m sure it will get better.

Accessibility Report: AngelList

For this accessibility report, I evaluated the website AngelList at Angel.co.

Angel.co is an information directory of startups in different cities like Seattle and San Francisco. Individual company pages feature information on founders, employees, and investors. Some startups also have a jobs page that feature open positions.

angel-home

The website was evaluated based on the Section 508 standard for accessibility, including the technical requirements for Web based intranet and Internet information and applications(1194.22).

1194.22 (a) A text equivalent for every non-text element shall be provided (e.g., via “alt”, “longdesc”, or in element content).

Requirement met? Yes
The website does a nice job of providing alt tags to describe images on their pages.

angel-img-alt-tag

1194.22 (b) Equivalent alternatives for any multimedia presentation shall be synchronized with the presentation.

Requirement met? Yes
Companies are able to post YouTube videos on their company page and closed captioning was available.

angel-video-cc

1194.22 (c) Web pages shall be designed so that all information conveyed with color is also available without color, for example from context or markup.

Requirement met? Yes
Hyperlinks display an underline when hovered over and buttons have a clear shape to indicate a clickable link.

angel-color

1194.22 (d) Documents shall be organized so they are readable without requiring an associated style sheet.

Requirement met? Yes
Yes, pages are readable with CSS styles disabled with Chrome’s developer tools extension.

angel-no-styles

1194.22 (e) Redundant text links shall be provided for each active region of a server-side image map.

Not applicable.

1194.22 (f) Client-side image maps shall be provided instead of server-side image maps except where the regions cannot be defined with an available geometric shape.

Not applicable.

1194.22 (g) Row and column headers shall be identified for data tables.

Requirement met? No
Some tables do not include

tags to indicate a column header.

angel-table-headers

1194.22 (h) Markup shall be used to associate data cells and header cells for data tables that have two or more logical levels of row or column headers.

Requirement met? Yes

No multi-level tables were found on the website.

1194.22 (i) Frames shall be titled with text that facilitates frame identification and navigation.

Requirement met? Yes
Frames are not used in the HTML of the pages.

1194.22 (j) Pages shall be designed to avoid causing the screen to flicker with a frequency greater than 2 Hz and lower than 55 Hz.

Requirement met? Yes
There is no detectable flicker during page load.

1194.22 (k) A text-only page, with equivalent information or functionality, shall be provided to make a web site comply with the provisions of this part, when compliance cannot be accomplished in any other way. The content of the text-only page shall be updated whenever the primary page changes.

Requirement met? No
A link to an alternative text-only page is not found on the site’s webpages.

Conclusion

The tables on the city pages could be made more accessible with the addition of table HTML tags. Also, there is no way to view a text-only version of any webpage. Otherwise, AngelList does a good job of making their site accessible based on the Section 508 standard.

Mobile Usability Test: EdX

For the mobile usability test I chose to review the mobile application for edx.org.

edx3

Edx is a MOOC or massively open online course website that aggregates courses from schools like Harvard and UC Berkeley and provides the course in an online format for free with an optional paid certificate. There are several courses on computer science and web development.

Demographic tested: Men and women aged 32-60.

Test subjects were asked to perform the following tasks while verbalizing their thoughts.

1. Sign up for a Edx account

2. Browse at least two computer science courses

3. Enroll is one computer science class

4. Watch a video lecture or read some course text

Test Results

Task 1:

Testers had no difficulty signing up for an account on a mobile device. The site presented a simple and intuitive options for signing up for an account including signing up with social accounts like Facebook. The form inputs were optimized for mobile devices. Once the testers were finished submitting their information they were presented with a message stating “You’re almost there!” that asks the user to check their email to confirm their account as the final step before enrolling in courses. This caused some confusion because you don’t have to check your email before enrolling in courses. Also it can be cumbersome to have to navigate away from the site to check your email before continuing.

Task 2:

Testers had no difficulty browsing computer science courses on the site. One tester scrolled through all the featured course previews on the main page instead of navigating to the computer science category, filtering, or searching.

Task 3:

Testers had no difficulty enrolling in a course and quickly found the “Enroll Now” button on each course description page. Once the tester enrolled in a course and navigated to the home dashboard or “My Courses” page the page was not optimized for mobile. The dashboard page does not fit the mobile screen so the user has to scroll right to see all the information.

Task 4:

One tester had difficulty watching a course video because after they navigated to the course page they could not find the link to the lectures. After several minutes they finally found a link at the top navigation of the course page titled “Course” which led to the course lecture videos.

Analysis and Suggestions

The course page provides a lot of useful information in bite sized chunks like a video introduction, the course description, frequently asked questions, and student reviews. The breadcrumbs provide a sense of location within the website.

edx1

Further down the page there is a terrific table that provides the most important information like price and difficulty in a scannable format.

edx2

Additionally when I play a video from the course, it automatically redirects me to my YouTube app on my phone. I was surprised that I was taken away from the website and it would have been nice if I received a warning or option to watch the video at the website.

edx5

The message at the end of the sign up process stating “You’re almost there!” and asking the user to confirm their email can cause confusion. It is not clear whether this step is actually required because testers were able to continue and enroll in courses. This step can also cause unnecessary difficulty for someone using a mobile device since it is more difficult to check their email and then return to the site.

After users have signed up for a course and navigate to the “My Courses” page they are presented with a dashboard page where the content does not fit the screen. Optimizing the dashboard page for mobile can improve usability of the app.

edx4

Once a user has enrolled in a course and navigated to the course page it is not obvious how to view the course material like video lectures. There is a small text link titled “Course” at the top of the page but it is easy to miss. This can be made more obvious and an additional button link can be added to the page body to improve usability of users attempting to view the course material.

Usability Test: Udacity

The website I chose for this report is Udacity.com which provides web development and computer science courses.

udacity

Test subjects were asked to perform the following four tasks while verbalizing their thoughts.

1. Show me how you would sign up for an account

2. Enroll in the free “Front End Frameworks” course

3. Watch a video lecture (you don’t have to watch the entire video)

4. Navigate to the home dashboard that displays your progress in your courses

Test purpose: To determine if new users can get started with Udacity courses without difficulty.

Test methods: Record screencasts of test subjects.

Test number and demographics: 3 individuals in the 20-60 age range.

Results

Task1: Show me how you would sign up for an account

The user had no problem finding the link at the top named “Sign Up.” There were only four fields so it was very obvious to the user that they had to input a first name, last name, email, and password. The user did not realize that she could have signed up using Facebook until after she had already signed up for an account.

Task2: Enroll in the free “Front End Frameworks” course

The user struggled initially to find the course titled “Front End Frameworks.” She eventually found a button titled “View Catalog” on the page “My Classroom.” Initially she commented that there were a lot of courses and started scrolling through the list. After a minute she realized she could use the search box and also filter by “Free Courses.”

Task3: Watch a video lecture (you don’t have to watch the entire video)

The first video started playing automatically so she had no trouble starting the first lecture video.

Task4: Navigate to the home dashboard that displays your progress in your courses

The user did not immediately realize how to navigate to the dashboard. She clicked on several links that did not take her to the dashboard such as “Resources” and the navigation menu icon. She finally clicked on “< Front End Frameworks" and it took her to the dashboard that listed courses she was enrolled in.

Next Steps

The user did not realize that she could sign up using Facebook but would have preferred that method rather than using an email and password. Perhaps more prominent text could announce that Facebook login is available.

udacity-7

The user could not immediately find the course “Front End Frameworks” and it took a minute to realize there is a search box at the top and filters on the left. The search box may be easy to miss because of the style of the input box with light colors and thin lines. This looks aesthetically pleasing but may cause users to not see that there is a search box available. A possible A/B test could be to make the search box stand out more and check how users respond.

udacity-8

The user had trouble finding the link to navigate to the dashboard that displays current classes. The user probably would have found it if my task described the dashboard as the homepage because there is a large home icon on the left. So I don’t consider that a problem with the UX. The user may or may not know that the current classes would be on the home page.

More Notes

The home page is focused primarily on getting people interested in the “Nanodegree” and it isn’t until three quarters down the home page that you find information about free courses.

udacity-1

The primary goal seems to be to get visitors to sign up for a nanodegree rather than to enroll in a free course as almost all of the home page content is about nanodegrees.

The about page succinctly describes their mission to bring affordable education to the masses.

udacity-2

When you click on the Nanodegree link on the homepage it brings you to a clean and well-organized page with more information about Nanodegrees in general and links to learn more about specific Nanodegrees.

udacity-3

I like that the content is scannable because it is broken up into small chunks of information. You can quickly learn about the most important points like that you will receive a 50% refund if you graduate within 12 months and you are guaranteed a job within 6 months with the Nanodegree Plus.

Udacity has chosen to make it difficult to learn the fact that the Nanodegree can be taken for free through the free courses. I believe the $199 per month is for support from instructors, a forum, and a certificate of achievement but the curriculum is the same.

This information is hidden and actually on a separate website (Zendesk).

udacity-4

This seems like a business decision to not advertise the free version more clearly but I still like that the curriculum is available for free.

Usable Design Analysis: Surf Incubator

For this usable design analysis I looked at the Surf Incubator website at http://www.surfincubator.com/. Surf Incubator is a startup incubator and co-working space in Seattle.

Criteria #1: Are navigational elements clear?

The gray navigation bar at the top of the homepage is obvious and intuitive. It has 7 top level navigation links to important topics like “Tour”, “Cafe”, and “Events.” Some of the navigation links have additional sublinks, which is made obvious by the arrow icon next to the navigation link. The navigation bar has a gray background which contrasts with the white background so it catches the eye of the user.

surf

Criteria #2: Is content presented with clear hierarchies?

Yes, the navigational menu provides clues to the organizational structure of the website’s pages. The drop down menus under the links “Community”, “Events”, and “Join Us” show that there is more information to read about within these categories.

Criteria #3: Do I know what I’m looking at on each page?

There is a heading at the top of each page describing what the page is about.

surf-2

On the Cafe page, there are two rows of photos of the cafe immediately following the heading. This shows the user what the cafe looks like but you have to scroll down to learn that the cafe is a membership plan where users receive free coffee and can work alongside 160 startup employees. This important information about what the cafe offers may be useful if added right after the heading.

The page title is also highlighted in the navigation menu based on the page the user is currently viewing.

Criteria #4: Can I see there is more content if I scroll down?

surf-3

The only indicator that there is more content below the fold is when the content is cut off abruptly.

On the Membership Levels page, there is too much white space between the first row and second row of membership levels. This can cause the user to mistakenly think that there is no more additional content lower on the page.

surf-4

Criteria #5: Do I know where I am in this website?

There are good cues to indicate where the user is on the website such as the large blue headings and the highlighting of the active tab.

There are no breadcrumbs on the site, however they are probably not necessary since the webpages only go two levels deep.

Clicking on the logo in the top left corner will return the user to the home page. The site could add a “Home” link to the navigation menu to provide users another path back to the home page.

Criteria #6: Am I presented with clear, easily understand content blocks?

The content blocks are kept to short paragraphs without any intimidating walls of text.

surf-5

One thing that could help usability is to use h2 headings above each paragraph to provide a way for users to scan for the information they are looking for. The above example uses headings that are centered and capitalized which is a subtle clue, but the headings could stand out more if they were bolded and a different color.