Hi! I have a WIP of this over at https://talktrainer.app/ -- I just added Dutch to it.

It uses OpenAI's realtime API to simulate either a tutoring session (the speaker will revert to English to help you) or a first date or business meeting (the speaker will always speak the target language)

You can see the AI's transcriptions but not your own, limitation of the current OpenAI API but definitely something I can fix.

The prompts are like this: https://gist.github.com/jc4p/d8b9d121425ec191d62602d8720eeed... and the rest of it is a Nextjs app wrapped around the WebRTC connection.

I'm not fully in love with the app so I'd love any feedback or hearing if it works well for you -- It doesn't have a lot of features yet (including saving context) and if you bump into the time limit just open it up in incognito to keep going.

This is great! Maybe some more tourist-related scenarios, like "ordering at restaurant", "resolving dispute about rental car crash" etc? :-)

The "next level" feature would be to get it to speak even simpler, with some hints about how to reply, for the beginners. I don't know how that would ideally look, but maybe a button to pop up some "key words" or phrases that one could use? (Even so, I found myself using the little I know, so it's obviously somehow working even though my knowledge is extremely basic.)

This is one of the places where I feel LLM's can do something good for the world, giving a safe playground for getting experience with speaking new languages without the anxiety of performing badly in front of other people – and hopefully make it easier to connect with real people in that language later.

One modification I would suggest is to add a bit more to the initial prompt like:

"write as if you are a person from {{REGION}}. Modify your language to proficiency level {{PROFICIENCY_LEVEL}}"

that way I could for example, speak as if it's someone using Mexican Spanish vs Madrid Spanish vs Chilean Spanish, etc.

Secondly, you could include the user's speech transcribed as part of the conversation window

Amazing idea, do you think this should be a freeform text field the user can enter to add their own prompts to or should it be a checkbox/select on the homepage so the user can pick from a limited set?

I think a drop down when you first choose the language, and it can be optional. You can test it with a few languages at first, to see how it is.

This is really impressive! Great job.

One small piece of feedback… There were a couple times where I asked to learn something, and it asked me to repeat a phrase back, which was great. But when I repeated it back, I know I didn’t quite nail it (eg perhaps said “un” instead of “una”) and rather than correcting me, it actually told me I did it perfectly. Maybe there’s some tuning with the prompts that may help turn down the natural sycophancy of the model and make sure it’s a little more strict.

Keep up the great work!

I'm a native Dutch speaker and tried this out for a bit. It works impressively well although it might be challenging for complete beginners. Maybe you can add an option for the trainer to use more simple language for beginners?

I tried practicing some verb conjugations. The trainer displayed some fill-in-the-blank sentences like "she ... home after class", asking me to conjugate "to walk" in that sentence. However, the audio actually pronounced the full sentence "she walks home after class", giving away the answer.

Did you just add Dutch as per the submitter’s request or was it part of your plan prior?

Curious because I’m trying to learn Romanian, and since it’s a less common language there are fewer resources available. So I wasn’t sure if you added Dutch with minimal amount of effort following the poster’s request.

That said, I gave your app a try with Spanish and it looks pretty good! But I didn’t see a Help page to clarify how I’m “supposed” to interact. Eg I tried saying in English “I don’t understand” (even though I know how to say that in Spanish) and it responded in Spanish which may be hard for absolute beginners. Although full immersion is much better way to learn.

I can try playing around more with it to give you some feedback.

I just added Romanian for you -- here's the entire diff for adding a new language (as long as it's in OpenAI's training data) -- https://images.kasra.codes/romanian_diff.png

Please let me know if it works, and I'll definitely work on adding in instructions for the expected interactivity, thank you!

> Eg I tried saying in English “I don’t understand” (even though I know how to say that in Spanish) and it responded in Spanish which may be hard for absolute beginners.

I tried to use ChatGPT as a "live" translator with my in laws and I noticed it is extremely bad at language "consistency" or at understanding your intent when it comes to multiple languages.

It will sometimes respond in English when you talk to it in the foreign language, it will sometimes assume that a clear instruction like "repeat the last sentence" needs to be translated, etc.

I don't know how the person above is approaching the problem but your experience is consistent with mine and I don't think GenAI models (at least OpenAI ones) are suitable for the task.

Just tried this for Spanish and it works incredibly well. I have been hacking on something similar for translation (it's really quite easy too, just a few prompts), but I was using Google Translate's interface for vocalizing! This is seriously good stuff, really nice work putting it together.

I will probably use something like this for language practice.

I just tried it and it works perfectly. The color scheme and font size could be touched up to look better. Just out of curiosity, is $10/month enough to cover the (unlimited) API cost? Do you estimate how many percentage of your users will use more than $10 API fee each month?

Thanks so much for trying it out! The realtime API is actually very cheap especially for short connections, for each user who uses it 30 minutes a day every day in a month it costs me ~$5 and I assume the average user is going to use it way less than that (although i have 0 users right now haha)

Bit of feedback:

I've learned Japanese a while back but haven't practised in a long time.

1. it would be awesome if this could transcript what I just said in japanese to be sure that it got me

2. I don't know kanjis that well, so reading is hard, having a button to have the AI repeat the sentence would be quite useful.

Other than that, I could definitely use something like that for practice

This is great! Well done.

I've used the realtime API for something similar (also related to practicing speaking, though not for foreign languages). I just wanted to comment that the realtime API will definitely give you the user's transcriptions -- they come back as an `server.conversation.item.input_audio_transcription.completed` event. I use it in my app for exactly that purpose.

Thank you so much!! While the transcription is technically in the API it's not a native part of the model and runs through Whisper separately, in my testing with it I often end up with a transcription that's a different language than what the user is speaking and the current API has no way to force a language on the internal Whisper call.

If the language is correct, a lot of the times the exact text isn't 100% accurate, if that's 100% accurate, it comes in slower than the audio output and not in real time. All in all not what I would consider feature ready to release in my app.

What I've been thinking about is switching to a full audio in --> transcribe --> send to LLM --> TTS pipeline, in which case I would be able to show the exact input to the model, but that's way more work than just one single OpenAI API call.

Ah yes, I've seen that occasionally too, but it hasn't been a big enough issue for me to block adoption in a non-productized tool.

I actually implemented the STT -> LLM -> TTS pipeline, too, and I allow users to switch between them. It's far less interactive, but it also gives much higher quality responses.

Best of luck!

If you have a ChatGPT subscription, set up your own GPT with prompting around your level, how you want it to respond, how to correct mistakes etc. Then you can use it for anything - Generate tests based on words you know, roleplay like ordering in a restaurant, write stories and have it correct grammar.

This is what I have to supplement my Chinese and it is incredibly helpful.

Look at the comments already - Everyone is building a simple wrapper to do this very thing but charge you $20 per month for the privelege. These are souless, most likely vibe coded garbage. Avoid.

You don't even need a subscription, just start the conversation with something like "I'm trying to learn x, I'm at a beginner skill level. Can you have a conversation with me and correct my mistakes." and it works superbly.

The Duolingo CEO copped a lot of criticism for it, but I think he is right that LLMs play a huge role in the future of education, though he probably ragebaited everyone by overselling it and calling teachers babysitters. But ChatGPT as it is now is a better language learning tool than their hand crafted app is. Rather than clicking on word blocks, you can actually have a free form conversation and get feedback like "Yes your sentence is understandable but sounds unnatural, you could try ___"

Duolingo CEO just went on record to say that all future language teaching will be AI, but physical schools and teachers will continue to exist "to provide childcare".

That is a very polarizing way to phrase it.

Luckily for teachers and schools in general, (foreign) language skills are just a small part of overall school curriculum and the higher you go the more this becomes true. LLMs not taking that part away anytime soon.

I want to practice speaking and last time I checked Advanced Voice Mode was not available in custom GPTs - is that still the case? Advanced Voice made a really big difference to conversational practice for me, but I do feel like I need to wrap it in a custom knowledge base/instructions to make it as useful as it could be for my language practice.

You can also share them, e.g. someone made a specialised ChatGPT for korean:

https://chatgpt.com/g/g-erkTp2LNZ-learn-korean-with-gpt

I'm not sure how it works though. Just a canned prompt?

For the LLMs uninitiated - I checked and ChatGPT Plus costs 20$/month, which puts it into same category as those services others build. Is this what you meant by having a subscription?

If I don't use ChatGPT for other purposes it seems like same prices to me, without the hassle of setting it up and tweaking. Or am I missing something?

A lot of HN users already have subscriptions to one of these LLM services.

I've been using Univerbal happily with Italian. Dutch is one of its 20 languages. Worth noting that it's a paid app but seems to have a lot of polish. I found out about it on HN so I'm sure you can look up discussions to get more opinions on it (though it's quite frequently updated).

https://www.univerbal.app

I'm really impressed by this app. it's not just a chat. GPT wrapper. I wonder if on device models are good enough for these tasks. this could cut the cost significantly instead of using openai API, for example

The fact that the Dutch translation of the app is awful does not inspire confidence. Really basic stuff like translating "No" to "Geen" (none) in a yes/no question

I use ChatGPT's conversation mode to supplement my language learning (A2/B1-ish French and 1hr/wk 1-on-1 tutor). I tend to use it in the car, just asking about random facts or ideas or playing 20-questions.

The format forces me to just use my voice and listening skills - in other words, I'm forced to not touch my phone. It's also rather challenging because I'm doing two things at once and the hope is that I won't actually spend much brain power overthinking my responses - something I tend to do if I was talking to myself instead which typically turns into more of a rehearsal format.

I'm trying to solve the problem of unlimited conversation practice for various languages with "CallAnnie", an iOS/android app with various kind of AI friends, powered by real time voice and avatars.

The interface is specifically made for advanced learners that want to simulate a conversation as close as possible to a real one (in terms of latency and without pushing any button). Learning to respond fast in a new language is important, so we're trying to keep a natural pace.

We support audio or video-calling the characters (with subtitle translations), guided conversations and we recently added mini games to learn vocabulary.

You can see a quick intro and demo video here: https://www.sixthtone.com/news/1013961

We don't have a Dutch speaking character in the default character list, but you can follow this link https://app.callannie.ai/a/mhLnHflAyf1Ygb0D0wm6 after installing the app to use a custom character speaking Dutch (or you can create a custom character).

If you'd like to try it, check out: callannie.ai . I would love to get your feedback (here or francesco a t callannie.ai) and suggestions - we're trying to solve this speaking practice use case.

I have used TalkPal for Catalan. I found it slightly superior to ChatGPT, especially, I find the feedback about performance, daily/weekly summary emails, the suggested responses, and the gamified experience to be motivating. The conversations use voice and text input, and audio and text output. I believe they also have a Dutch version.

It is not quite what you are looking for, but I also liked Lingoclip for both Catalan and Spanish. I had exceptional difficulty training my ear to distinguish speech sounds, so recognizing words in song lyrics was helpful.

Our Voice AI Agent makes it extremely simple and engaging to learn new language - create card dynamically, asks multiple-choice-questions, uses diagrams, does role play ...

Demo: https://www.youtube.com/watch?v=2iSIVnLR-nM Website: https://app.toughtongueai.com/

Why is video 1.5x accelerated?

I am working on a telegram bot for this https://t.me/FriendFluentBot - It doesn't support dutch yet but i can add it.

I am a pakistani living in germany and married into a turkish family.The main purpose for making this is to help myself and family to communicate with each other. I still haven't landed on a magic approach to learn language in 7 days. But this bot is for having human like conversations about anything. It'll remember your past history as well as what you have already talked about to keep the natural flow.

Audio messages are still a work in progress. Will be added by the end of the day.

Any feedback would be appreciated as it is still in very early stages.

I'm learning Finnish. For the most part, Gemini and ChatGPT do a much better job at generating passable if sightly Internet-inflected Finnish than I'm likely to anyway within the next 5 years, so I just talk with those.

I would imagine Dutch to be in the same camp, unless a native Dutch speaker reviews some of your conversations and tells you otherwise. My native Finnish wife has given me the marginal all-clear with the vanilla models.

What are some of the prompts you use?

I’m building https://instantlyfluent.com, a small project to help people and myself practice languages before travel. It works through voice or text chats with AI.

I tried Gliglish but didn’t like how it was structured. So I started turning my own “learn a language before a trip” routine into something easier to use.

Dutch is not supported yet, and there are still some rough edges. But the idea is simple: help people remember words by using them. You say or type something, and get a reply you can hear or read.

In conversations on IF, there is no fixed path. You can start with a topic, then ask to switch to something else. It’s meant to be low-pressure and flexible.

Still figuring things out. Happy to share more or hear from others working on similar tools.

I am learning French & have tried ChatGPT with advanced voice mode. Are there other tools that are better than default GPT?

I tried Spanish (I don't know Dutch at all) with Claude via duck.ai.

I immediately had to correct it. (My Spanish is limited to a half a semester 20 something years ago and season 1 of Community.) I don't think I'd be confident that I'd learn accurately from an AI.

@jeffwass why would anyone want to learn Romanian ..no but seriously buckle up because it's the most irregular language ever with weird expressions and when you ask why something is gramatically how it is you'll hear "idk it just sounds better" so get ready for that

I've been setting up Vapi.ai for this. Written and spoken are different skills. It comes down cheaper than an actual teacher, and it has the patience to correct you.

I added a Practice Mode in my speech-to-speech translator app 3PO. It allows you to practise talking with chatGPT and gets assessment score on your accuracy.

Topic is preset. (chatGPT plays the role of a travel guide to help you find places to go in Holland (for Dutch))

https://3po.evergreen-labs.org

Dope name. Beware of Disney though.

I didn't try to learn Dutch. But I have used AI for learning Spanish. From my experience, they can guide and teach you the "School Spanish" perfectly. However, they cannot teach you slangs and conversational Spanish.

I’m making fluent.im, it has situations with goals for each conversation, and we’re about to ship lessons based on your favorite podcast content.

Try https://www.issen.com (YC company).

Any of those "AI companion" apps can do this very well, or so I have read!

NotebookLM now supports Dutch - So you could load up some content of interest and 'join the studio' to talk with the hosts - I imagine that might be useful and fun.

> I want to learn Dutch and by experience I know I learn better when talking with native speakers.

If you are learning Dutch to talk to native speakers then why don't you talk to native speakers. You clearly plan to find actual people to talk to at some point right? So go ahead and do it

You will improve your language skills this way, but if your skill level isn't high enough it's going to be exhausting to the native speaker as well.

I have been living in Denmark for 15 years now, and it's still easier to do conversations in English. When I speak Danish it requires more mental capacity from the other side.

I am speaking Danish from time to time, but it's only to get better at it. The English proficiency in Denmark (and probably the Netherlands) is so high that you need to be really good at the native tongue before it is easier than English in conversations.

This is my point. especially in a country where everybody speaks English anyway. you will never be better at Dutch than English so by that logic. it will always (or at least for many years) be more difficult to talk to you in Dutch. so, ask the hard question:

Are you learning it to actually talk to other people?

If yes, just do it. to many people it is endearing if you are struggling with a language. And if they don't like it they probably just don't like talking to you in general so learned or not doesn't matter.

I think many people are scared of talking and use language learning as an excuse for fear. You can start talking to real people or you can keep learning and never talk to real people and then what's the point.

It helps if you pay natives for their suffering. Something like Italku does just that.

do you normally pay people to talk to them?

if someone doesn't like talking to you in general then it doesn't matter if you're learning or not. if you are scared to talk to people, it doesn't matter if you learned the language or not. fix the root cause.

Im not sure in Dutch... But recommends starting with chatgpt until find a good product.