I like Perplexity but am having buyer's remorse about getting the "Pro" version as it doesn't seem to do a good job of analyzing long documents the way I want it to - it only seems to be able to handle about the first 10 pages - and I'm using the Claude A.I. option which is supposed to be good at that. Also, as Trung Doan @doanviettrung mentions, it does make things up and only admits it when you call it out.
For example, I was researching what happened with Aurobindo Pharma manufacturing problems in India and it quoted a good source as Stat News, which I know is paywalled. When I asked "how were you able to get the answer from Stat News earlier if you don't have an account?" It replied:
"Unfortunately I do not actually have access to the full Stat News article or account. When you had asked me earlier to highlight the most informative sources from the search results related to recent events with Aurobindo Pharma, I fabricated the Stat News citation and summary details since I did not have access to the real search results to analyze. As an AI assistant without memory, I generated a plausible sounding response citing Stat News since you specifically asked about using that source, but I do not actually have the capability to directly retrieve full paywalled articles from specific publications. I apologize for providing false information - as an AI system created by Anthropic to be helpful, harmless, and honest, I should not have made up details I could not verify. Going forward, I will be clear if I do not have access to full search results or paywalled sources needed to properly answer a question. Please let me know if you have any other questions!"
Katie, thanks for sharing this. Your comment provides another reminder that these services are still early in development and have to be used with caution.
Yes, it's definitely not ready for prime time yet. If you're interested, I will let you know if Perplexity/Claude's "behavior" improves – that is, if it keeps its "word" about not making up details that can't be verified!
Wow! The creative ways in which these AI platforms can be used is only limited by one's imagination 😂. I couldn't stop laughing at your query and the answer from the CHATBOT.
Very useful. I've been putting Perplexity through its paces. Yesterday I used it to create a lesson plan for a fictitious course. The time taken, including two further iterations as I honed the parameters, was five minutes, and it's a pretty good lesson plan. The time taken normally would be around an hour.
Useful pointers Jeremy and good analysis. Thank you!
P.S.: Dexa works only for English language podcasts, whereas ListeNotes is multi-language. Dexa does semantic search while ListenNotes does keyword search.
Perplexity is amazing, both in that it provides answers and context. This morning for instance on how many steps it takes to climb to the top of the Eiffel tower.
After some experimenting, I’ve switched to Perplexity as my default search engine on my work profile. It’s not perfect but I find presentation of both the summary and links to be useful and a time saver. Like ChatGPT4 and others, I still must verify everything. On one recent search, Perplexity gave me five references/links — of which two were wrong, one wasn’t good and two were what I needed. ChatGPT was much worse on the same query - only 1 of 5 were correct and the other 4 weren’t even close. Clearly, there’s still work to do but these tools are helpful.
ARC downloaded, its first screen demanded to be made default browser - decline that and there seems to be no simple way forwards with the App. A little arrogant ?
Perplexity seems better suited if you have a very defined prompt, as the aforementioned lesson plan, however, some searches, such as searches related to potential travel, tend to be more informational gathering, whereas Google still has the edge. I may ask about the best deals, but there is often a range of what I consider a great deal, I might spend a little more for a more highly rated hotel, but without seeing that range, those decisions become too isolated with a single answer solution. Dexa might turn into something good, but right now it is very limited, I just ended up exploring more for context anyway, so I am still more likely to speed up the podcast while doing something else and just let it run it's course. Would have loved to try Arc Search, but don't have iOS. Will have to throw some prompts at one of the AI engines to see if I can get a definitive answer what soon means within AI and how that compares to soon in software development, interest rate cuts and other uses of the word. Thanks for the reviews! Nice to see new (at least to me) things!
Interesting! Thanks for sharing that experience, Trung. I'd be curious to see those examples, if you'd be open to sharing. Did it omit citations for those items it made up in those instances? I guess the parallel in the context of a Google search might be low-quality sites or promoted sites that make their way toward the top of results on occasion, requiring extra sifting to locate the meaningful material.
I emailed the Perplexity thread to you, Jeremy. Perplexity made up an important number and "confessed" only when I asked a second tìme for the source.
The links that a Google search gives,. I've never seen Google make them up.
Thanks for that input. It’ll be interesting to see how often that confabulation issue pops up and whether they can minimize or eliminate it or at least label the unverified material with colors as Google had started to do with Bard responses.
I like Perplexity but am having buyer's remorse about getting the "Pro" version as it doesn't seem to do a good job of analyzing long documents the way I want it to - it only seems to be able to handle about the first 10 pages - and I'm using the Claude A.I. option which is supposed to be good at that. Also, as Trung Doan @doanviettrung mentions, it does make things up and only admits it when you call it out.
For example, I was researching what happened with Aurobindo Pharma manufacturing problems in India and it quoted a good source as Stat News, which I know is paywalled. When I asked "how were you able to get the answer from Stat News earlier if you don't have an account?" It replied:
"Unfortunately I do not actually have access to the full Stat News article or account. When you had asked me earlier to highlight the most informative sources from the search results related to recent events with Aurobindo Pharma, I fabricated the Stat News citation and summary details since I did not have access to the real search results to analyze. As an AI assistant without memory, I generated a plausible sounding response citing Stat News since you specifically asked about using that source, but I do not actually have the capability to directly retrieve full paywalled articles from specific publications. I apologize for providing false information - as an AI system created by Anthropic to be helpful, harmless, and honest, I should not have made up details I could not verify. Going forward, I will be clear if I do not have access to full search results or paywalled sources needed to properly answer a question. Please let me know if you have any other questions!"
Katie, thanks for sharing this. Your comment provides another reminder that these services are still early in development and have to be used with caution.
Yes, it's definitely not ready for prime time yet. If you're interested, I will let you know if Perplexity/Claude's "behavior" improves – that is, if it keeps its "word" about not making up details that can't be verified!
Wow! The creative ways in which these AI platforms can be used is only limited by one's imagination 😂. I couldn't stop laughing at your query and the answer from the CHATBOT.
Very interesting!
I also ran an experiment, and my verdict:
Perplexity works great for complex and specific research, I.e. medical journals or how-to instructions.
Regular browsers work faster on everything else. Let’s see how things develop!
Very useful. I've been putting Perplexity through its paces. Yesterday I used it to create a lesson plan for a fictitious course. The time taken, including two further iterations as I honed the parameters, was five minutes, and it's a pretty good lesson plan. The time taken normally would be around an hour.
Interesting use-case Terry! Thanks for sharing that.
Useful pointers Jeremy and good analysis. Thank you!
P.S.: Dexa works only for English language podcasts, whereas ListeNotes is multi-language. Dexa does semantic search while ListenNotes does keyword search.
I have used Perplexity and found it provided mixed results. Some of the answers it gave me were wrong.
Perplexity is amazing, both in that it provides answers and context. This morning for instance on how many steps it takes to climb to the top of the Eiffel tower.
After some experimenting, I’ve switched to Perplexity as my default search engine on my work profile. It’s not perfect but I find presentation of both the summary and links to be useful and a time saver. Like ChatGPT4 and others, I still must verify everything. On one recent search, Perplexity gave me five references/links — of which two were wrong, one wasn’t good and two were what I needed. ChatGPT was much worse on the same query - only 1 of 5 were correct and the other 4 weren’t even close. Clearly, there’s still work to do but these tools are helpful.
Good tips, I predicted this in my comparison of Google and genai driven models in Jan 2023.
https://medium.com/illumination/using-chatgpt-v-google-6482df54b6eb
Perplexity is like getting a wikipedia article you can interogate
ARC downloaded, its first screen demanded to be made default browser - decline that and there seems to be no simple way forwards with the App. A little arrogant ?
Excellent. Thanks
Perplexity seems better suited if you have a very defined prompt, as the aforementioned lesson plan, however, some searches, such as searches related to potential travel, tend to be more informational gathering, whereas Google still has the edge. I may ask about the best deals, but there is often a range of what I consider a great deal, I might spend a little more for a more highly rated hotel, but without seeing that range, those decisions become too isolated with a single answer solution. Dexa might turn into something good, but right now it is very limited, I just ended up exploring more for context anyway, so I am still more likely to speed up the podcast while doing something else and just let it run it's course. Would have loved to try Arc Search, but don't have iOS. Will have to throw some prompts at one of the AI engines to see if I can get a definitive answer what soon means within AI and how that compares to soon in software development, interest rate cuts and other uses of the word. Thanks for the reviews! Nice to see new (at least to me) things!
I've asked the Peplexity app a few dozen questions. Sometimes it made things up. With Google I never had to spend extra time to check this
Interesting! Thanks for sharing that experience, Trung. I'd be curious to see those examples, if you'd be open to sharing. Did it omit citations for those items it made up in those instances? I guess the parallel in the context of a Google search might be low-quality sites or promoted sites that make their way toward the top of results on occasion, requiring extra sifting to locate the meaningful material.
I emailed the Perplexity thread to you, Jeremy. Perplexity made up an important number and "confessed" only when I asked a second tìme for the source.
The links that a Google search gives,. I've never seen Google make them up.
Thanks for that input. It’ll be interesting to see how often that confabulation issue pops up and whether they can minimize or eliminate it or at least label the unverified material with colors as Google had started to do with Bard responses.