Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed CSV agent not working with OpenAI GPT #2393

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

Hariharan1828
Copy link
Contributor

Pull Request Description:

This pull request addresses an issue #2341 encountered with the CSV agent in Flowise when processing CSV data containing prefixed column names. The issue manifested as an error preventing successful retrieval of information from the CSV file, specifically when querying for city data.

Changes Made:

Regex Function Implementation:
    Introduced a removePrefix function utilizing a regular expression pattern to remove any prefixes from column names. The regex pattern effectively targets and removes prefixes added by OpenAI to the column names in the CSV data.

Map Function Application:
    Applied the removePrefix function to each node's label property using the map function. This ensures that any prefixes present in the labels of the starting nodes are removed before they are added to the activeChatflows object.

Logger Info Enhancement:
    Added logging statements to provide visibility into the addition and removal of chatflows from the ChatflowPool. These statements aid in tracking the execution flow and monitoring the status of chatflows.

Expected Outcome:

With these changes, the prefixes from column names in the CSV data will be effectively removed, enabling the CSV agent to accurately process the data without encountering errors. This should resolve the issue described in the bug report, allowing for successful retrieval of city data from CSV files.

If there are any further questions or adjustments needed, please feel free to discuss them in the comments.

@Hariharan1828
Copy link
Contributor Author

Opps, there is a lint error, let me fix

@@ -17,8 +17,19 @@ export class ChatflowPool {
* @param {ICommonObject} overrideConfig
*/
add(chatflowid: string, endingNodeData: INodeData | undefined, startingNodes: IReactFlowNode[], overrideConfig?: ICommonObject) {
const removePrefix = (colName: string) => colName.replace(/^"[^"]*"/, '') // Regex to remove prefix
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the suggested fix!

why do we have the fix here though? I thought it should be in the CSVAgent.ts line 178 to check if the pythonCode contains any prefix

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought of changing it directly from the Pool because not only CSV AGENT AirtableAgent also had same issue, but yes I do find changing from CSV AGENT is precise, will revert back and do he checks in both CSV and AirtableAgent

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants