For Zoho Services only:


I'm actually part of a bigger team at Ascent Business Solutions where we have support technicians and project consultants. Support is for smaller technical fixes but this can include developments, reports or integrations; depending on the size of the task. Projects are for more time-consuming developments such as revamps of the Zoho Suite of apps or on-site training. The advantage of a team is that if I am out-of-office for a day or so, there is always someone at Ascent Business Solutions who can deal with any queries/issues you may have.

Our support rates can be found and purchased at http://ascentbusiness.co.uk/zoho-support-2. A support bundle doesn't have an expiry date. So whether we can do what you want within the bundle and a year later need further support, if there are minutes left on the bundle then there is no additional charge.

Our project rates for bigger developments can be found at http://ascentbusiness.co.uk/crm-solutions/zoho-crm-packages-prices and will involve a dedicated project consultant along with developers who will hold your hand through the development process.

If you want help building a solution for one of the Zoho Apps in the Zoho Suite, contact us on 0121 293 8140 (UK) or by email at info@ascentbusiness.co.uk. You can also visit our website at http://ascentbusiness.co.uk.

I regularly build and specialize in 2-way API integrations for Xero, Shopify and eBay.

Zoho Deluge: Handle Commas between Quotes in a CSV (and New Lines)

What?
This is an article to demonstrate how to handle commas in a CSV file that were enclosed between two double-quotes. I've added to this the handling of new lines or carriage returns in between a pair of double-quotes.

Why?
Our use case is that we were trying to loop through rows of a CSV file which contained addresses which in turn contained commas. Saving this as a CSV and asking Deluge to parse the data in the appropriate columns was not working as expected.

How?
The quick answer is a regex that will replace any commas between two quotes with a custom string, to be exact:
copyraw
v_FormattedData = r_Data.replaceAll("(\"[^\",]+)[,]([^\"]+\")","$1|mySpecialComma|$2",false);
  1.  v_FormattedData = r_Data.replaceAll("(\"[^\",]+)[,]([^\"]+\")","$1|mySpecialComma|$2",false)

The slightly longer answer to describe this might be better explained if you consider the following snippet of code:
copyraw
// sample data
v_Test = "00011,Joel Lipman,\"Flat 8, House Corner\",Brummieland";

// regex to replace a comma found between 2 double-quotes to a string of your choice
v_FormattedString = v_Test.replaceAll("(\"[^\",]+)[,]([^\"]+\")","$1|mySpecialComma|$2",false);

info v_FormattedString;
// yields: 00011,Joel Lipman,"Flat 8|mySpecialComma| House Corner",Brummieland

// split into a list (string delimited by commas)
l_StringParts = v_FormattedString.toList();

// show me column 3 with myspecialcomma replaced back to a comma
info l_StringParts.get(2).replaceAll("|mySpecialComma|", ",", true);

// yields: Flat 8, House Corner
  1.  // sample data 
  2.  v_Test = "00011,Joel Lipman,\"Flat 8, House Corner\",Brummieland"
  3.   
  4.  // regex to replace a comma found between 2 double-quotes to a string of your choice 
  5.  v_FormattedString = v_Test.replaceAll("(\"[^\",]+)[,]([^\"]+\")","$1|mySpecialComma|$2",false)
  6.   
  7.  info v_FormattedString; 
  8.  // yields: 00011,Joel Lipman,"Flat 8|mySpecialComma| House Corner",Brummieland 
  9.   
  10.  // split into a list (string delimited by commas) 
  11.  l_StringParts = v_FormattedString.toList()
  12.   
  13.  // show me column 3 with myspecialcomma replaced back to a comma 
  14.  info l_StringParts.get(2).replaceAll("|mySpecialComma|", ",", true)
  15.   
  16.  // yields: Flat 8, House Corner 

The long answer is to consider the following code which generates a sample CSV and then loops through storing each row as a data record and outputting it to screen:
copyraw
// generate a sample CSV file
v_DataCSV = "1,Joel Lipman,Kings Castle,England\n";
v_DataCSV = v_DataCSV + "11,General Dogsbody,\"Flat 8, House Corner, My Street\",Brummieland\n";
v_DataCSV = v_DataCSV + "64,Elephant Man,\"123 New Street,\nNew York City\",USA\n";
f_CSVfile = v_DataCSV.toFile("test.csv");
//
// read and process CSV file
v_FileContent = f_CSVfile.getFileContent();
l_FileRows = List();
if(!isBlank(v_FileContent))
{
	l_FileRows = v_FileContent.toList("\n");
}
// loop through each row
for each r_Data in l_FileRows
{
	// initialize record
	m_Record = Map();
	m_Record.put("EmployeeID","");
	m_Record.put("Name","");
	m_Record.put("Address","");
	m_Record.put("Territory","");
	//
	// split values by commas
	l_FieldValues = r_Data.toList(",");
	if(l_FieldValues.size()>0)
	{
		m_Record.put("EmployeeID",l_FieldValues.get(0));
	}
	if(l_FieldValues.size()>1)
	{
		m_Record.put("Name",l_FieldValues.get(1));
	}
	if(l_FieldValues.size()>2)
	{
		m_Record.put("Address",l_FieldValues.get(2));
	}
	if(l_FieldValues.size()>3)
	{
		m_Record.put("Territory",l_FieldValues.get(3));
	}
	info m_Record.toString();
}
  1.  // generate a sample CSV file 
  2.  v_DataCSV = "1,Joel Lipman,Kings Castle,England\n"
  3.  v_DataCSV = v_DataCSV + "11,General Dogsbody,\"Flat 8, House Corner, My Street\",Brummieland\n"
  4.  v_DataCSV = v_DataCSV + "64,Elephant Man,\"123 New Street,\nNew York City\",USA\n"
  5.  f_CSVfile = v_DataCSV.toFile("test.csv")
  6.  // 
  7.  // read and process CSV file 
  8.  v_FileContent = f_CSVfile.getFileContent()
  9.  l_FileRows = List()
  10.  if(!isBlank(v_FileContent)) 
  11.  { 
  12.      l_FileRows = v_FileContent.toList("\n")
  13.  } 
  14.  // loop through each row 
  15.  for each r_Data in l_FileRows 
  16.  { 
  17.      // initialize record 
  18.      m_Record = Map()
  19.      m_Record.put("EmployeeID","")
  20.      m_Record.put("Name","")
  21.      m_Record.put("Address","")
  22.      m_Record.put("Territory","")
  23.      // 
  24.      // split values by commas 
  25.      l_FieldValues = r_Data.toList(",")
  26.      if(l_FieldValues.size()>0) 
  27.      { 
  28.          m_Record.put("EmployeeID",l_FieldValues.get(0))
  29.      } 
  30.      if(l_FieldValues.size()>1) 
  31.      { 
  32.          m_Record.put("Name",l_FieldValues.get(1))
  33.      } 
  34.      if(l_FieldValues.size()>2) 
  35.      { 
  36.          m_Record.put("Address",l_FieldValues.get(2))
  37.      } 
  38.      if(l_FieldValues.size()>3) 
  39.      { 
  40.          m_Record.put("Territory",l_FieldValues.get(3))
  41.      } 
  42.      info m_Record.toString()
  43.  } 

Without a regex solution and the replacement back to commas then the above outputs:
copyraw
{"EmployeeID":"1","Name":"Joel Lipman","Address":"Kings Castle","Territory":"England"}
{"EmployeeID":"11","Name":"General Dogsbody","Address":"\"Flat 8","Territory":" House Corner"}
{"EmployeeID":"64","Name":"Elephant Man","Address":"\"123 New Street","Territory":""}
{"EmployeeID":"New York City\"","Name":"USA","Address":"","Territory":""}
  1.  {"EmployeeID":"1","Name":"Joel Lipman","Address":"Kings Castle","Territory":"England"} 
  2.  {"EmployeeID":"11","Name":"General Dogsbody","Address":"\"Flat 8","Territory":" House Corner"} 
  3.  {"EmployeeID":"64","Name":"Elephant Man","Address":"\"123 New Street","Territory":""} 
  4.  {"EmployeeID":"New York City\"","Name":"USA","Address":"","Territory":""} 

Actions
  • Do you want to get rid of the annoying escaped double-quotes that no longer have any use in our data sample? Use a non-regex replaceall:
    copyraw
    v_FormattedData = v_FormattedData.replaceAll("\"","",true);
    1.  v_FormattedData = v_FormattedData.replaceAll("\"","",true)
  • If you are splitting your CSV rows by "\n", then you might want to consider that not all new lines/line breaks/carriage returns are at the end of a line.... for example, maybe your CSV contains multi-line fields; eg. where address street is on a different line to address city but in the same column... The following regex will replace new lines found in between 2 double-quotes and not at the end of the line:
    copyraw
    v_FileContent = v_FileContent.replaceAll("(\"[^\"\n]*)\r?\n(?!(([^\"]*\"){2})*[^\"]*$)","$1|mySpecialNewLine|",false);
    1.  v_FileContent = v_FileContent.replaceAll("(\"[^\"\n]*)\r?\n(?!(([^\"]*\"){2})*[^\"]*$)","$1|mySpecialNewLine|",false)
  • Replace all commas in between a pair of double-quotes: Well I couldn't find a single regex that can do this in one go but by applying the same regex to the same value (line 29), solved this for the sample data. If you are expecting a 3rd comma in the value, then apply the regex again.

The Final Solution:
The following code is a correction to the above and replaces any commas between two double-quotes with a custom string and then replaces this back to a comma when encountered. It will also replace any new lines found between two double-quotes with a single space (note how I've added a new line in the third row in the sample data):
copyraw
// generate a sample CSV file 
 v_DataCSV = "1,Joel Lipman,Kings Castle,England\n"; 
 v_DataCSV = v_DataCSV + "11,General Dogsbody,\"Flat 8, House Corner, Street\",Brummieland\n"; 
 v_DataCSV = v_DataCSV + "64,Elephant Man,\"123 New Street,\nNew York City\",USA\n"; 
 f_CSVfile = v_DataCSV.toFile("test.csv"); 
 // 
 // read and process CSV file 
 v_FileContent = f_CSVfile.getFileContent(); 
 l_FileRows = List(); 
 if(!isBlank(v_FileContent)) 
 { 
     // replace the new line between two quotes with |mySpecialNewLine| 
	 v_FileContent = v_FileContent.replaceAll("(\"[^\"\n]*)[\r\n](?!(([^\"]*\"){2})*[^\"]*$)","$1|mySpecialNewLine|",false); 
     l_FileRows = v_FileContent.toList("\n"); 
 } 
 // loop through each row 
 for each r_Data in l_FileRows 
 { 
     // initialize record 
     m_Record = Map(); 
     m_Record.put("EmployeeID",""); 
     m_Record.put("Name",""); 
     m_Record.put("Address",""); 
     m_Record.put("Territory",""); 
     // 
     // replace the comma between two quotes with |mySpecialComma| 
	 v_FormattedData = r_Data.replaceAll("(\"[^\",]+)[,]([^\"]+\")","$1|mySpecialComma|$2",false); 
	 // again if there could be another comma in the value (repeat if more commas expected)
	 v_FormattedData = v_FormattedData.replaceAll("(\"[^\",]+)[,]([^\"]+\")","$1|mySpecialComma|$2",false); 
	 // replace any double-quotes 
     v_FormattedData = v_FormattedData.replaceAll("\"","",true); 
     // 
     // split values by commas 
     l_FieldValues = v_FormattedData.toList(","); 
     if(l_FieldValues.size()>0) 
     { 
         v_ThisColumnValue = l_FieldValues.get(0).trim(); 
         v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialComma|", ",", true); 
         v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialNewLine|", " ", true); 
         m_Record.put("EmployeeID",v_ThisColumnValue); 
     } 
     if(l_FieldValues.size()>1) 
     { 
         v_ThisColumnValue = l_FieldValues.get(1).trim(); 
         v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialComma|", ",", true); 
         v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialNewLine|", " ", true); 
         m_Record.put("Name",v_ThisColumnValue); 
     } 
     if(l_FieldValues.size()>2) 
     { 
         v_ThisColumnValue = l_FieldValues.get(2).trim(); 
         v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialComma|", ",", true); 
         v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialNewLine|", " ", true); 
         m_Record.put("Address",v_ThisColumnValue); 
     } 
     if(l_FieldValues.size()>3) 
     { 
         v_ThisColumnValue = l_FieldValues.get(3).trim(); 
         v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialComma|", ",", true); 
         v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialNewLine|", ", ", true); 
         m_Record.put("Territory",v_ThisColumnValue); 
     } 
     info m_Record.toString(); 
 }
  1.  // generate a sample CSV file 
  2.   v_DataCSV = "1,Joel Lipman,Kings Castle,England\n"
  3.   v_DataCSV = v_DataCSV + "11,General Dogsbody,\"Flat 8, House Corner, Street\",Brummieland\n"
  4.   v_DataCSV = v_DataCSV + "64,Elephant Man,\"123 New Street,\nNew York City\",USA\n"
  5.   f_CSVfile = v_DataCSV.toFile("test.csv")
  6.   // 
  7.   // read and process CSV file 
  8.   v_FileContent = f_CSVfile.getFileContent()
  9.   l_FileRows = List()
  10.   if(!isBlank(v_FileContent)) 
  11.   { 
  12.       // replace the new line between two quotes with |mySpecialNewLine| 
  13.       v_FileContent = v_FileContent.replaceAll("(\"[^\"\n]*)[\r\n](?!(([^\"]*\"){2})*[^\"]*$)","$1|mySpecialNewLine|",false)
  14.       l_FileRows = v_FileContent.toList("\n")
  15.   } 
  16.   // loop through each row 
  17.   for each r_Data in l_FileRows 
  18.   { 
  19.       // initialize record 
  20.       m_Record = Map()
  21.       m_Record.put("EmployeeID","")
  22.       m_Record.put("Name","")
  23.       m_Record.put("Address","")
  24.       m_Record.put("Territory","")
  25.       // 
  26.       // replace the comma between two quotes with |mySpecialComma| 
  27.       v_FormattedData = r_Data.replaceAll("(\"[^\",]+)[,]([^\"]+\")","$1|mySpecialComma|$2",false)
  28.       // again if there could be another comma in the value (repeat if more commas expected) 
  29.       v_FormattedData = v_FormattedData.replaceAll("(\"[^\",]+)[,]([^\"]+\")","$1|mySpecialComma|$2",false)
  30.       // replace any double-quotes 
  31.       v_FormattedData = v_FormattedData.replaceAll("\"","",true)
  32.       // 
  33.       // split values by commas 
  34.       l_FieldValues = v_FormattedData.toList(",")
  35.       if(l_FieldValues.size()>0) 
  36.       { 
  37.           v_ThisColumnValue = l_FieldValues.get(0).trim()
  38.           v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialComma|", ",", true)
  39.           v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialNewLine|", " ", true)
  40.           m_Record.put("EmployeeID",v_ThisColumnValue)
  41.       } 
  42.       if(l_FieldValues.size()>1) 
  43.       { 
  44.           v_ThisColumnValue = l_FieldValues.get(1).trim()
  45.           v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialComma|", ",", true)
  46.           v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialNewLine|", " ", true)
  47.           m_Record.put("Name",v_ThisColumnValue)
  48.       } 
  49.       if(l_FieldValues.size()>2) 
  50.       { 
  51.           v_ThisColumnValue = l_FieldValues.get(2).trim()
  52.           v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialComma|", ",", true)
  53.           v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialNewLine|", " ", true)
  54.           m_Record.put("Address",v_ThisColumnValue)
  55.       } 
  56.       if(l_FieldValues.size()>3) 
  57.       { 
  58.           v_ThisColumnValue = l_FieldValues.get(3).trim()
  59.           v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialComma|", ",", true)
  60.           v_ThisColumnValue = v_ThisColumnValue.replaceAll("|mySpecialNewLine|", ", ", true)
  61.           m_Record.put("Territory",v_ThisColumnValue)
  62.       } 
  63.       info m_Record.toString()
  64.   } 
Yields:
copyraw
{"EmployeeID":"1","Name":"Joel Lipman","Address":"Kings Castle","Territory":"England"}
{"EmployeeID":"11","Name":"General Dogsbody","Address":"Flat 8, House Corner, Street","Territory":"Brummieland"}
{"EmployeeID":"64","Name":"Elephant Man","Address":"123 New Street, New York City","Territory":"USA"}
  1.  {"EmployeeID":"1","Name":"Joel Lipman","Address":"Kings Castle","Territory":"England"} 
  2.  {"EmployeeID":"11","Name":"General Dogsbody","Address":"Flat 8, House Corner, Street","Territory":"Brummieland"} 
  3.  {"EmployeeID":"64","Name":"Elephant Man","Address":"123 New Street, New York City","Territory":"USA"} 

Source(s):
Category: Zoho :: Article: 759

Credit where Credit is Due:


Feel free to copy, redistribute and share this information. All that we ask is that you attribute credit and possibly even a link back to this website as it really helps in our search engine rankings.

Disclaimer: The information on this website is provided without warranty and any content is merely the opinion of the author. Please try to test in development environments prior to adapting them to your production environments. The articles are written in good faith and, at the time of print, are working examples used in a commercial setting.

Thank you for visiting and, as always, we hope this website was of some use to you!

Kind Regards,

Joel Lipman
www.joellipman.com

Related Articles

Joes Revolver Map

Accreditation

Badge - Certified Zoho Creator Associate
Badge - Certified Zoho Creator Associate

Donate & Support

If you like my content, and would like to support this sharing site, feel free to donate using a method below:

Paypal:
Donate to Joel Lipman via PayPal

Bitcoin:
Donate to Joel Lipman with Bitcoin - Valid till 8 May 2022 3QnhmaBX7LQSRsC9hh6Je9rGQKEGNQNfPb
© 2021 Joel Lipman .com. All Rights Reserved.