Client-Side Development

On the client, we want to add the following

Voice Recognition
HTTP Requests to the server

Voice Recognition

HTML5 contains voice recognition APIs that we will be using.

Create a file called index.js in the www folder.

Check if browser supports Voice Recognition

!('webkitSpeechRecognition' in window) means that the window object does not have speech recognition.

Start Recognizing

Create the recognition object var recognition = new webkitSpeechRecognition(); recognition.interimResults = true; // enables recognition results to change before finalizing
Deal with different callback functions

We need to have different functions to handle different events triggered by the Recognition API.

Let's go ahead and add dummy functions for the callbacks.

recognition.onstart = function() {
    console.log("started");
}
recognition.onresult = function(event) {
    console.log("onResult");
}
recognition.onerror = function(event) {
    console.log(event);
}
recognition.onend = function() {
}

Info on events: onstart - called when recognition starts. onresult - called when words have been made out. onerror - called when error is triggered. onend - called when recognition has ended.

onresult is probably our most important.

We have two variables that are very important: final_transcript and interim_transcript.

final_transcript is the collection of all important words and interim_transcript is the collection of words that have been detected but haven't been finalized yet.

Getting the results

In onresult we want to first get the result

We can get the result by iterating through event.results.length. We need to start at event.resultIndex because the recognized index keeps increasing along with the size.

var final_transcript = '';
recognition.onresult = function() {
    var interim_transcript = '';
    for(var i = event.resultIndex; i < event.results.length; i++) {

    }
}

Next, if it's finalized, add it to the final_transcript variable. Otherwise, add to the interim_transcript variable.

for(var i = event.resultIndex; i < event.results.length; i++) {
    if(event.results[i].isFinal) {
        final_transcript += event.results[i][0].transcript;
    } else {
       interim_transcript += event.results[i][0].transcript; 
   }
}

The HTML

Create your index.html in the www folder. Start it out with the standard:

<html>
    <body>
        <script src="main.js"></script>
    </body>
</html>

Next, let's add a button with an onclick listener:

    ...
    <body>
        <button onclick="startRecognition()"> Start Recognizing </button>
    </body>
    ...

Then, we need to create that listener:

function startRecognition() {
    final_transcript = ''; // we want to reset the transcript each time we start recognition
    recognition.lang = 'en-US';
    recognition.start();
}

Also, let's add a div with two spans.

<div>
    <span id="final_span" class="final"></span>
    <span id="interim_span" class="interim"></span>
</div>

Let's create a styles.css with the following:

.interim {
    color: gray;
}

This is so that we can differentiate between the interim and final results.

Let's add the interim and final results to the screen.

var finalSpan = document.getElementById('final_span');
var interimSpan = document.getElementById('interim_span');

...
recognition.onresult = function(event) {
    var interim_transcript = '';
    for(var i = event.resultIndex; i < event.results.length; i++) {
        if(event.results[i].isFinal) {
            final_transcript += event.results[i][0].transcript;
        } else {
            interim_transcript += event.results[i][0].transcript;
        }
    }
    fanSpan.innerHTML = final_transcript;
    interimSpan.innerHTML = interim_transcript;
    console.log("Final", final_transcript);
    console.log("Interim", interim_transcript);
}
...

Go ahead and add a stop button to stop the recognition:

<button onclick="stopRecognition()"> Stop Recognition </button>

function stopRecognition() {
    recognition.stop();
}

Making requests

We basically want to make requests once the recognition stops, so you guys learned the fetch command, well let's go ahead and use it! To search for an artist, the command would be search for artist <artistName> We need to:

Check if the voice recognition command contains search and for artist
Parse the string for the artist

fetch using the artist

recognition.onend = function() {
     if(final_transcript.toLowerCase().indexOf('search') > -1) {
         if(final_transcript.toLowerCase().indexOf('for artist') > -1) {
             let index = final_transcript.toLowerCase().indexOf('artist');
             let artist = final_transcript.substr(index + 7);
             fetch(`http://localhost:3000/search/artist/${artist}`).then(res => res.json()).then(data => console.log(data));
        }
     }
}

Next, we need to display that info, so let's add a place to display on the html:

<div id="search_results">
    <h3 id="artist_name"></h3>
    <img id="artist_image">
</div>

Then, we need to add the html to add the data to the different elements

fetch(`http://localhost:3000/search/artist/${artist}`).then(r => r.json()).then(data => {
    var artist = data.items[0];
    console.log(artist);
    var name_header = document.getElementById('artist_name');
    name_header.innerHTML = artist.name;
    var artist_image = document.getElementById('artist_image');
    artist_image.src = artist.images[0].url;
});

Then, you can do the same for searching for a song!

Client-Side