React Native speech-to-text functionality is a common use case for developers today. Whether it's general use or accessibility, the need for speech-to-text in the project is likely to arise at some point, and it's something we as developers should be prepared to implement in our applications.
This article will show you how to build a speech-to-text dictation application using React Native.
-
prerequisites
-
installation and setup
-
Build the user interface
-
Home/index.tsx
-
Notes/index.tsx
-
-
Integrated Speech Recognition Library
-
hooks/useCreateNote.ts
-
hooks/useNotes.ts
-
final presentation
prerequisites
-
node.js
-
android studio
-
Xc Carol
-
vs code
To run apps on iOS, you need macOS and Xcode to compile and run the simulator.
For Android, you can use the Android Emulator in Android Studio, or simply connect an Android device to run it. Also, how to download and update IE browser? Just Three Easy Steps We'll also use VS Code to build the app.
installation and setup
There are two typical approaches when building mobile applications with React Native.
One uses Expo - a suite of tools built around React Native designed to improve development productivity. Another way is to use React Native CLI, which is basically like starting from scratch, without any toolset supporting React Native development.
In this tutorial, we'll be using Expo so the development process is more manageable.
let's start:
npm install -g expo-cli
After installing the Expo CLI globally, you can initialize the project with the following command:
expo init <Name of Project>
Here we will choose a blank project with TypeScript configuration. You can also create a JavaScript version with some predefined functions if you want.
Once it scaffolds the project, you can run the app for Android or iOS. Navigate to that directory and run one of the following npm commands:
- cd SpeechToTextAppDemo - npm run android - npm run ios - npm run web
How to fix Lenovo power manager not working as you can see in the folder structure ? How to open the power manager? is the entry point of the application. is an expo configuration that configures how the project loads and generates rebuilds for Android and iOS. App.tsx``app.json
Build the user interface
Shown above is a simple wireframe that we will use to build the app's UI.
We keep it simple because we want to focus on functionality - once we've built the app, you can customize it however you want and practice on it.
Let's first build the navigation in the application.
To implement navigation in a React Native application, we need to install the following packages:
npm install @react-navigation/bottom-tabs @react-navigation/native @expo/vector-icons
Since we'll be implementing navigation at the bottom, we'll need the install and core packages. @react-navigation/bottom-tabs``@react-navigation/native
Also, to add support for icons and text we need that package. @expo/vector-icons
For navigation, there are three components:
-
NavigationContainer
-
Tab.Navigator
-
Tab.Screen
Wraps everything inside. Fun Notes help navigate between different components while rendering the components themselves. NavigationContainer Tab.NavigatorTab. Screen
Now, we will change the function including navigation: App.tsx
import { StatusBar } from "expo-status-bar";
import { FontAwesome5 } from "@expo/vector-icons";
import { StyleSheet, Text, View } from "react-native";
import { createBottomTabNavigator } from "@react-navigation/bottom-tabs";
import { NavigationContainer } from "@react-navigation/native";
import Home from "./components/Home";
import Notes from "./components/Notes";
const Tab = createBottomTabNavigator();
export default function App() {
return (
<NavigationContainer>
<Tab.Navigator
screenOptions={({ route }) => ({
tabBarIcon: () => {
let iconName = "record";
if (route.name === "Record") {
iconName = "record-vinyl";
} else if (route.name === "History") {
iconName = "history";
}
return <FontAwesome5 name={iconName} size={24} color="black" />;
},
})}
>
<Tab.Screen name="Record" component={Home} />
<Tab.Screen name="History" component={Notes} />
</Tab.Navigator>
</NavigationContainer>
);
}
const styles = StyleSheet.create({
container: {
flex: 1,
backgroundColor: "#fff",
alignItems: "center",
justifyContent: "center",
},
});
Create and component by adding the following code: Home``Notes
Home/index.tsx
import { useState, useEffect } from "react";
import {
View,
Text,
TextInput,
StyleSheet,
Button,
Pressable,
} from "react-native";
export default function Home() {
return (
<View style={styles.container}>
<Text>Home Screen</Text>
</View>
);
}
const styles = StyleSheet.create({
container: {},
});
Notes/index.tsx
import { useState, useEffect } from "react";
import {
View,
Text,
TextInput,
StyleSheet,
Button,
Pressable,
} from "react-native";
export default function Notes() {
return (
<View style={styles.container}>
<Text>Notes Screen</Text>
</View>
);
}
const styles = StyleSheet.create({
container: {},
});
Now, we have a navigation bar at the bottom of the application. Let's run the application and see how it works. To run the app, do one of the following:
-
It will build and run the app expo start in the iOS and Android emulators using the command
-
It will build the project for iOS and Android, create native code in the project directory, and run it with the command npx expo run:<ios|Android>
Here we'll follow the second approach, sitemap since we have a library that needs to interact with custom native code. Using commands is easier and simpler when you don't have such a requirement, since we don't need to manage native code builds ourselves. voice``expo start
You will see the following error when you run the app with the command: npx expo run:ios
This error is due to the use of some core utilities that we skip installing during setup. So let's install them and re-run the application: Navigation``Navigation
npm install react-native-safe-area-context react-native-gesture-handler react-native-screens react-native-web
Now, we have screen navigation in our application. So let's integrate a library for speech-to-text functionality in our application. voice
Integrated Speech Recognition Library
First, install the library in your application like so: react-native-voice
npm i @react-native-voice/voice --save
After installing the npm package, add the configuration plugin to the plugins array: app.json
{
"expo": {
"plugins": ["@react-native-voice/voice"]
}
}
Then, add permission in configuration: app.json
"ios": {
"supportsTablet": true,
"bundleIdentifier": "com.anonymous.SpeectToTextApp",
"infoPlist": {
"NSSpeechRecognitionUsageDescription": "This app uses speech recognition to convert your speech to text.",
"NSCameraUsageDescription": "This app uses the camera to let user put a photo in his profile page."
}
},
"android": {
"adaptiveIcon": {
"foregroundImage": "./assets/adaptive-icon.png",
"backgroundColor": "#FFFFFF"
},
"permissions": ["android.permission.RECORD_AUDIO"],
"package": "com.anonymous.SpeectToTextApp"
},
Let's build a UI that integrates with the library in the component. react-native-voice``Home
Before we get in, the UI needs some animation for the record/stop functionality. So let's install and implement it: react-native-reanimated `` @motify/components
npm i @motify/components react-native-reanimated
Add plugin to Babel configuration: react-native-reanimated
module.exports = function (api) {
api.cache(true);
return {
presets: ["babel-preset-expo"],
plugins: ["react-native-reanimated/plugin"],
};
};
After installation, implement speech recognition in: react-native-reanimated``Record/index.tsx
import React, { Component } from "react";
import { FontAwesome } from "@expo/vector-icons";
import { MotiView } from "@motify/components";
import {
StyleSheet,
Text,
View,
Image,
TouchableHighlight,
} from "react-native";
import Voice, {
SpeechRecognizedEvent,
SpeechResultsEvent,
SpeechErrorEvent,
} from "@react-native-voice/voice";
import { Easing } from "react-native-reanimated";
type Props = {
onSpeechStart: () => void;
onSpeechEnd: (result: any[]) => void;
};
type State = {
recognized: string;
pitch: string;
error: string;
end: string;
started: boolean;
results: string[];
partialResults: string[];
};
class VoiceTest extends Component<Props, State> {
state = {
recognized: "",
pitch: "",
error: "",
end: "",
started: false,
results: [],
partialResults: [],
};
constructor(props: Props) {
super(props);
Voice.onSpeechStart = this.onSpeechStart;
Voice.onSpeechRecognized = this.onSpeechRecognized;
Voice.onSpeechEnd = this.onSpeechEnd;
Voice.onSpeechError = this.onSpeechError;
Voice.onSpeechResults = this.onSpeechResults;
Voice.onSpeechPartialResults = this.onSpeechPartialResults;
Voice.onSpeechVolumeChanged = this.onSpeechVolumeChanged;
}
componentWillUnmount() {
Voice.destroy().then(Voice.removeAllListeners);
}
onSpeechStart = (e: any) => {
console.log("onSpeechStart: ", e);
this.setState({
started: true,
});
};
onSpeechRecognized = (e: SpeechRecognizedEvent) => {
console.log("onSpeechRecognized: ", e);
this.setState({
recognized: "√",
});
};
onSpeechEnd = (e: any) => {
console.log("onSpeechEnd: ", e);
this.setState({
end: "√",
started: false,
});
this.props.onSpeechEnd(this.state.results);
};
onSpeechError = (e: SpeechErrorEvent) => {
console.log("onSpeechError: ", e);
this.setState({
error: JSON.stringify(e.error),
});
};
onSpeechResults = (e: SpeechResultsEvent) => {
console.log("onSpeechResults: ", e);
this.setState({
results: e.value!,
});
};
onSpeechPartialResults = (e: SpeechResultsEvent) => {
console.log("onSpeechPartialResults: ", e);
this.setState({
partialResults: e.value!,
});
};
onSpeechVolumeChanged = (e: any) => {
console.log("onSpeechVolumeChanged: ", e);
this.setState({
pitch: e.value,
});
};
_startRecognizing = async () => {
this.setState({
recognized: "",
pitch: "",
error: "",
started: false,
results: [],
partialResults: [],
end: "",
});
try {
await Voice.start("en-US");
this.props.onSpeechStart();
} catch (e) {
console.error(e);
}
};
_stopRecognizing = async () => {
try {
await Voice.stop();
} catch (e) {
console.error(e);
}
};
_cancelRecognizing = async () => {
try {
await Voice.cancel();
} catch (e) {
console.error(e);
}
};
_destroyRecognizer = async () => {
try {
await Voice.destroy();
} catch (e) {
console.error(e);
}
this.setState({
recognized: "",
pitch: "",
error: "",
started: false,
results: [],
partialResults: [],
end: "",
});
};
render() {
return (
<View style={styles.container}>
{this.state.started ? (
<TouchableHighlight onPress={this._stopRecognizing}>
<View
style={
{
width: 75,
height: 75,
borderRadius: 75,
backgroundColor: "#6E01EF",
alignItems: "center",
justifyContent: "center",
}}
>
{[...Array(3).keys()].map((index) => {
return (
<MotiView
from={
{ opacity: 1, scale: 1 }}
animate={
{ opacity: 0, scale: 4 }}
transition={
{
type: "timing",
duration: 2000,
easing: Easing.out(Easing.ease),
delay: index * 200,
repeatReverse: false,
loop: true,
}}
key={index}
style={[
StyleSheet.absoluteFillObject,
{ backgroundColor: "#6E01EF", borderRadius: 75 },
]}
/>
);
})}
<FontAwesome name="microphone-slash" size={24} color="#fff" />
</View>
</TouchableHighlight>
) : (
<TouchableHighlight onLongPress={this._startRecognizing}>
<View
style={
{
width: 75,
height: 75,
borderRadius: 75,
backgroundColor: "#6E01EF",
alignItems: "center",
justifyContent: "center",
}}
>
<FontAwesome name="microphone" size={24} color="#fff" />
</View>
</TouchableHighlight>
)}
</View>
);
}
}
const styles = StyleSheet.create({
button: {
width: 50,
height: 50,
},
container: {},
welcome: {
fontSize: 20,
textAlign: "center",
margin: 10,
},
action: {
textAlign: "center",
color: "#0000FF",
marginVertical: 5,
fontWeight: "bold",
},
instructions: {
textAlign: "center",
color: "#333333",
marginBottom: 5,
},
stat: {
textAlign: "center",
color: "#B0171F",
marginBottom: 1,
},
});
export default Record;
@react-native-voice` provides a class with functionality to start and stop voice recording and recognition. Some important methods are: `Voice
-
Voice.start("en-US");
-
Voice.stop();
-
Voice.cancel();
-
Voice.destroy();
Here we have two main functions: and -- these are the ones that handle the start and stop of the speech recognition function. startRecognizing`` stopRecognizing
Another important function to note is that it passes the speech result as text to the function via props. onSpeechEnd
onSpeechEnd = (e: any) => {
console.log("onSpeechEnd: ", e);
this.setState({
end: "√",
started: false,
});
this.props.onSpeechEnd(this.state.results);
};
After this, we import that speech component: Record``Home/index.tsx
import { useState, useEffect } from "react";
import {
View,
Text,
TextInput,
StyleSheet,
Button,
Pressable,
} from "react-native";
import Record from "../Record";
export default function Home() {
const [speechText, setSpeechText] = useState("");
return (
<View style={styles.container}>
<View style={styles.inputContainer}>
<Text style={styles.label}>Speech Text</Text>
<TextInput
multiline
style={styles.textInput}
numberOfLines={6}
value={speechText}
maxLength={500}
editable={true}
/>
<View
style={
{
alignItems: "flex-end",
flex: 1,
flexDirection: "row",
justifyContent: "space-between",
}}
>
<Button
title="Save"
color={"#007AFF"}
onPress={async () => {
console.log("save");
}}
/>
<Button
title="Clear"
color={"#007AFF"}
onPress={() => {
setSpeechText("");
}}
/>
</View>
</View>
<View style={styles.voiceContainer}>
<Record
onSpeechEnd={(value) => {
setSpeechText(value[0]);
}}
onSpeechStart={() => {
setSpeechText("");
}}
/>
</View>
</View>
);
}
const styles = StyleSheet.create({
container: {
flex: 1,
flexDirection: "column",
justifyContent: "center",
alignItems: "center",
width: "100%",
backgroundColor: "#F5FCFF",
},
label: {
fontWeight: "bold",
fontSize: 15,
paddingTop: 10,
paddingBottom: 10,
},
inputContainer: {
height: "50%",
width: "100%",
flex: 1,
padding: 10,
justifyContent: "center",
},
textInput: {
padding: 10,
borderColor: "#d1d5db",
borderWidth: 1,
height: 200,
borderRadius: 5,
},
saveButton: {
right: 0,
},
voiceContainer: {
height: "50%",
width: "100%",
alignItems: "center",
justifyContent: "space-around",
},
});
Now, we can access the results in . Let's implement the save function to store it in the database. We will use the following command for API mocking with a fake JSON server: speechText``Home/index.tsx
json-service -watch db.json
Create and add structure for it: db.json
{
"notes": []
}
We will use it for data fetching and API calls in our application. react-query
npm install react-query axios
To create and fetch notes, we can create custom hooks to handle queries and mutations:
hooks/useCreateNote.ts
import { QueryClient, useMutation } from "react-query";
import axios from "axios";
const createNote = async (note: string) => {
const { data } = await axios.post("http://localhost:3000/notes", {
note,
});
return data;
};
const useCreateNote = () =>
useMutation(createNote, {
onSuccess: (response) => {
},
});
export default useCreateNote;
hooks/useNotes.ts
import { useQuery } from "react-query";
import axios from "axios";
const fetchNotes = async () => {
const { data } = await axios.get("http://localhost:3000/notes");
return data;
};
const useNotes = () => useQuery("notes", fetchNotes);
export default useNotes;
Add hooks inside. useCreateNote``Home/index.tsx
import { useState, useEffect } from "react";
import {
View,
Text,
TextInput,
StyleSheet,
Button,
Pressable,
} from "react-native";
import { useMutation, useQueryClient } from "react-query";
import useCreateNote from "../../hooks/useCreateNote";
import Record from "../Record";
export default function Home() {
const [speechText, setSpeechText] = useState("");
const { mutate, isError, isLoading, isSuccess } = useCreateNote();
const queryClient = useQueryClient();
useEffect(() => {
if (isSuccess) {
setSpeechText("");
queryClient.invalidateQueries(["notes"]);
}
}, [isSuccess]);
return (
<View style={styles.container}>
<View style={styles.inputContainer}>
<Text style={styles.label}>Speech Text</Text>
<TextInput
multiline
style={styles.textInput}
numberOfLines={6}
value={speechText}
maxLength={500}
editable={true}
/>
<View
style={
{
alignItems: "flex-end",
flex: 1,
flexDirection: "row",
justifyContent: "space-between",
}}
>
<Button
title="Save"
color={"#007AFF"}
onPress={async () => {
console.log("save");
try {
await mutate(speechText);
} catch (e) {
console.log(e);
}
}}
/>
<Button
title="Clear"
color={"#007AFF"}
onPress={() => {
setSpeechText("");
}}
/>
</View>
</View>
<View style={styles.voiceContainer}>
<Record
onSpeechEnd={(value) => {
setSpeechText(value[0]);
}}
onSpeechStart={() => {
setSpeechText("");
}}
/>
</View>
</View>
);
}
const styles = StyleSheet.create({
container: {
flex: 1,
flexDirection: "column",
justifyContent: "center",
alignItems: "center",
width: "100%",
backgroundColor: "#F5FCFF",
},
label: {
fontWeight: "bold",
fontSize: 15,
paddingTop: 10,
paddingBottom: 10,
},
inputContainer: {
height: "50%",
width: "100%",
flex: 1,
padding: 10,
justifyContent: "center",
},
textInput: {
padding: 10,
borderColor: "#d1d5db",
borderWidth: 1,
height: 200,
borderRadius: 5,
},
saveButton: {
right: 0,
},
voiceContainer: {
height: "50%",
width: "100%",
alignItems: "center",
justifyContent: "space-around",
},
});
Add the following code inside: Note/index.tsx
import React from "react";
import {
View,
StyleSheet,
FlatList,
TouchableOpacity,
Text,
} from "react-native";
import useNotes from "../../hooks/useNotes";
export const Posts = ({}) => {
const { data, isLoading, isSuccess } = useNotes();
console.log(data);
return (
<View style={styles.container}>
{isLoading && (
<React.Fragment>
<Text>Loading...</Text>
</React.Fragment>
)}
{isSuccess && (
<React.Fragment>
<Text style={styles.header}>All Notes</Text>
<FlatList
data={data}
style={styles.wrapper}
keyExtractor={(item) => `${item.id}`}
renderItem={({ item }) => (
<TouchableOpacity onPress={() => {}} style={styles.post}>
<View style={styles.item}>
<Text style={styles.postTitle}>{item.note}</Text>
</View>
</TouchableOpacity>
)}
/>
</React.Fragment>
)}
</View>
);
};
const styles = StyleSheet.create({
container: {
flex: 1,
// backgroundColor: colors.white,
padding: 10,
},
wrapper: {
flex: 1,
paddingVertical: 30,
},
item: {
paddingVertical: 10,
paddingHorizontal: 20,
},
header: {
textAlign: "center",
textTransform: "capitalize",
fontWeight: "bold",
fontSize: 30,
// color: colors.primary,
paddingVertical: 10,
},
post: {
// backgroundColor: colors.primary,
padding: 15,
borderRadius: 10,
marginBottom: 20,
},
postTitle: {
color: "#000",
textTransform: "capitalize",
},
});
export default Posts;
Here we use the hook to get the data and render them in .useNotes``FlatList
in conclusion
In this article, we covered how to build a speech-to-text dictation application using React Native.
When you're building mobile apps that require hardware access or access to core libraries, it's important to understand how to access these resources with a mobile framework like React Native.
We've looked at these in detail in this article, and I hope you now have a better understanding of how to implement them in future projects. You can find the full source code here.