Forums

Additional rows added to MySQL

bensonley 07 Jul, 2019
HI - not really sure this is a bug with CF, but trying to analyse a recent and very annoying occurrence. This might not be the right forum area, but not sure where else to post this.

It's been going on for about 2-3 weeks now, a problem where my forms are suddenly adding additional rows with duplicate data, except for the aid field, upon submit.
I've not changed the forms since before it started happening, I even rolled back to a snapshot of the forms before additional rows were being added, and the problem still persists. I thought it was user error by double clicking, but the problem occurred for me also. Some forms that are duplicating have Loop events, some don't however, so it's not that it's misconfigured, as it would have been happening some time ago and, probably, fairly straight forward to spot and resolve.

Other things I've tried / investigated:

Tried various versions of CF / CC on the live and test sites.
Check LAMP log files for errors and found nothing related.
Rebooted server and ran any updates ( it didn't need any).


The weird thing is It's happening on most forms, but doesn't happen every time. The only pattern I can find is that it happens less often when the site is less busy, but it's not conclusive enough as it still happens when I'm the only person logged in using the site. Server resources are barely being used, so no loss of memory / disk space / CPU is averaging at 1%.

I'm a little baffled at this point, so any help on where or what to check, or if others have experienced this and resolved it, would be massively useful.

Some of the forms are complex as they have been slowly tweaked over the last 18 months as the business using them has grown quickly and many variables added.



Server details:
Plesk Onyx - Version 17.8.11 #53
‪Ubuntu 14.04.6 LTS
php 7.0.33 and mysql 5.5.62 nginx/1.11.10‪Ubuntu 14.04.6 LTS

I am willing to pay someone for investigating if they have some time to give up.
Thanks for reading.
bensonley 07 Jul, 2019
One thing to mention is that I have some forms on the same website within CC pages, and they haven't added additional rows once.
Max_admin 09 Jul, 2019
Hi bensonley,

Does your form have some security field ? better to use Google reCaptcha and make sure it works as expected, this issue is sometimes caused by other plugins loading the form page after its sent.

Best regards
Max, ChronoForms developer
ChronoMyAdmin: Database administration within Joomla, no phpMyAdmin needed.
ChronoMails simplifies Joomla email: newsletters, logging, and custom templates.
bensonley 09 Jul, 2019
Thanks Max. No security fields anywhere, all access to forms is restricted to specific usergroup levels. Should I add one to test?

Sadly it's pretty much a vanilla Joomla installation, extensions enabled are Akeeba CF CC and JCE editor. All up to date.

It's just weird that it sometimes happens, hard to find any pattern.
bensonley 09 Jul, 2019
Here's a log example - as you can see it's submitted twice, 100% not a double click.
2019-07-09 10:04:33 Access90.152.7.86303 POST /client-bookings-area/find-asignee/edit-asignee?aid=1350&chronoform=edit_asignee&event=submit HTTP/1.0585 Apache SSL/TLS access
2019-07-09 10:04:33 Access90.152.7.86303 POST /client-bookings-area/find-asignee/edit-asignee?aid=1350&chronoform=edit_asignee&event=submit HTTP/1.0585 Apache SSL/TLS access

I'll try disabling Akeeba and JCE see if that makes any difference.
bensonley 09 Jul, 2019
After further testing with user accounts. I am the only user the additional rows aren't added for.
I've changed access levels to other accounts to be the same ( Super Users) for testing and again weirdly the problem persists for them, even with the same access.
This the only pattern I've found so far.
healyhatman 09 Jul, 2019
You said you have a loop? Is the save data action in there?
bensonley 09 Jul, 2019
Yes inside the loop is a save data action. This saves the repeater area data into a table (Service Days). The rest of the form is saved to a separate table (Jobs). Happy to share login access and / or form(s).
healyhatman 09 Jul, 2019
so which table is getting doubled?
bensonley 09 Jul, 2019
There are 2 forms , one is to edit an existing Job and the other to add a new one.

The New Job form doubles for both Service Days ( saves inside the loop) and Jobs ( saves outside the loop)
The Edit Job form only doubles Service Days ( saves inside the loop )
bensonley 09 Jul, 2019
Sorry incorrect.

The New Job form doubles only Jobs ( saves outside the loop) but the Service Days ( inside the loop ) is as expected.
The Edit Job form only doubles Service Days ( saves inside the loop )
healyhatman 09 Jul, 2019
So have you set the edit ones to update, instead of insert?
bensonley 09 Jul, 2019
All the Services are within a repeater area inside the edit Job form.
There maybe existing Services pulled from a table related to the Job number. which appear within the repeater as expected.
They update fine if there is an aid value in the aid field inside the loop.
If they add an additional Service by clicking the multiply button within the repeater field, there is no aid value and the save action therefore adds a new one ( insert).
Maybe an issue with naming the field aid within the repeater? I could try changing it but doesn't explain to me) why it works fine for my login and no-one elses regardless of the ACL.
bensonley 09 Jul, 2019
Actually this afternoon some users can add a new Job on fine without doubling entries, it seems to work fine sometimes and not others. So it's not just my super user account that works fine all the time. Very confusing not to find a pattern.
Max_admin 10 Jul, 2019
Please try to add a honeypot field to one of the forms and test, if you have 6.1 then adding the field and enabling the "check security fields" will be enough for testing this.

Let us know!

Best regards
Max, ChronoForms developer
ChronoMyAdmin: Database administration within Joomla, no phpMyAdmin needed.
ChronoMails simplifies Joomla email: newsletters, logging, and custom templates.
bensonley 10 Jul, 2019
Thanks Max. Still using CFv6 classic at the moment.
Added honeypot to the New Job form and works fine for me as it did anyway, but doubled entry for a different user with or without any data in the repeater area ( Services). Also tried using CFv6.1 on a test environment (same server, on a subdomain), same thing.
Then tried again to re-check. This time the user added a new Job and it went on only once, no replication. Same user, same access, again with honeypot enabled.face-meh-blank
bensonley 10 Jul, 2019
here's a debug of one that worked fine using my login: ( now removed)
bensonley 10 Jul, 2019
This Job was added with 2 x Service Days - which were saved to a separate table inside a loop.
healyhatman 10 Jul, 2019
I highly doubt it's a big I think you've done your loops or save data wrong
Max_admin 10 Jul, 2019
Please try with an email or "message" action just after the loop with the save, do you receive the email more than one time also or a only once ?
Max, ChronoForms developer
ChronoMyAdmin: Database administration within Joomla, no phpMyAdmin needed.
ChronoMails simplifies Joomla email: newsletters, logging, and custom templates.
bensonley 10 Jul, 2019
Here's a debug of the edit Job form which duplicated the process for user Jamie, i know if was duplicated because it added data to the log table twice. All that was done was opened the edit Job form and hit submit.

Log screenshot:


Both user Ben (me) and Jamie did the same thing. Edit Job form - just hit submit. His is duplicated the action, mine hasn't.
My debug:
removed
healyhatman 10 Jul, 2019
So that's super long and too hard to follow just PM me access details and I'll glance at it.
bensonley 10 Jul, 2019
Yes it's a big form and had many changes over time, I'll probably have to rebuild them soon and split the Services and Jobs in the near future moving forward.
I've send over test server login details, all forms are the same as the live site, but data is slightly out of date, but same server & same LAMP environment
healyhatman 10 Jul, 2019
<?php $point = $this->get('loop_event36.row.meeting_point');
if(!$point) { $point = 'TBC'; }

echo $point;

?>

Instead of echoing from custom code blocks to get the variables, you can return from a PHP block. Example:
return $this->get('loop_event36.row.meeting_point') ?: 'TBC';

For your save data action, you're using "Auto Detect" which has NEVER reliably worked for me. Either use "Insert - Duplicate key update" or have a Read Data action set to First Matching looking for a row already existing. In the "Found" block have your save data action set to update. In the "Not found" block put a save data action set to insert.
healyhatman 10 Jul, 2019
For reference, I recommend the second option.
bensonley 10 Jul, 2019
OK many thanks - I'll make changes you've recommended,run some tests and post any results.
bensonley 11 Jul, 2019
I've now made changes but still occasionally adds a second row.

Its happening on a few forms, not just ones with a Loop or repeater area. Add POI for instance, pretty straight forward process. Saves to POI table and then Log table. But more often than not it creates a second row. See below

Apache Log:
2019-07-11 11:43:56 Access 90.152.7.86200 POST /client-bookings-area/find-asignee/add-poi?asignee_id=1271&event=submit&chronoform=add_poi HTTP/1.0 5.06 K Apache SSL/TLS access
2019-07-11 11:43:56 Access 90.152.7.86200 POST /client-bookings-area/find-asignee/add-poi?asignee_id=1271&event=submit&chronoform=add_poi HTTP/1.0 5.06 K Apache SSL/TLS access

Nothing in error log.

I've attached the POI form [file=https://klippa-relo.com/test/Add POI_11_Jul_2019_12_15_41.cf6bak]POI backup CF6[/file]
And here's a data snapshot related to that Job and apache log:
​[file=https://klippa-relo.com/test/Add POI_11_Jul_2019_12_15_41.cf6bak]POI Table[/file]
Any ideas?face-meh-blank
Max_admin 13 Jul, 2019
Did you run the test I suggested earlier to check an email or a message gets duplicated too or not ?

Also your debug has some private data I guess ? maybe you should remove that or delete the whole debug from your earlier post ?
Max, ChronoForms developer
ChronoMyAdmin: Database administration within Joomla, no phpMyAdmin needed.
ChronoMails simplifies Joomla email: newsletters, logging, and custom templates.
bensonley 13 Jul, 2019
Hi Max,

Yes sorry. Email was sent twice when the additional row was added, only once when not. However, system message only ever shows up once ( not duplicated), when an additional row is added or not.

Thanks, I've removed the debug for now.
healyhatman 13 Jul, 2019
Then the loop is happening twice and your loop is set up wrong.
bensonley 13 Jul, 2019
There is no loop in this form (POI) though.
healyhatman 13 Jul, 2019
The add_poi form? Which save data action is adding twice?
healyhatman 13 Jul, 2019
And does it happen when you preview it instead of looking at it on the front end?
bensonley 13 Jul, 2019
save_data3 ( saves to Poi table as insert) and save_data17 ( saves to log table as insert) - which is inside the save_data3 success body
bensonley 13 Jul, 2019
Yes, both do. like you would expect if there was a double click on save. This also happens in the preview (within admin template)
bensonley 29 Jul, 2019
Still going on. What ive done since.

replicated website on a fresh new server, using Centos instead of Ubuntu.
Used standard template - beez.
Rebuilt entire forms.

Problem still persists, now started happening on CC forms too.

Most of the time a form is submitted twice on submit, but not every time.

Next I'm trying a completely new Joomla CF CC installation with no other plugins installed. I'll let you know how i get on.
bensonley 09 Aug, 2019
Tried on a fresh installation on a separate new server and fresh Joomla and CC/C installation. Uploaded forms and added data to tables.

Problem still persists,form is sometimes Posted twice.

2019-08-09 11:47:07 Access 86.8.200.9200 POST /manager-jobs-view?status=complete&event=invoice&conn=manager-jobsheet&assignee_id=1503HTTP/1.03.96 KApache SSL/TLS access
2019-08-09 11:47:07 Access 86.8.200.9200 POST /manager-jobs-view?status=complete&event=invoice&conn=manager-jobsheet&assignee_id=1503HTTP/1.03.96 KApache SSL/TLS access

It looks like a double submit, but it's not. Only happens sometimes not all the time. Sometimes more often than not. Would there be an issue in the JS somewhere? Any ideas of where to test next? I'm running out of options and ideas at the moment which would ultimately mean rewriting all forms and views as a custom component and moving away from CC/CF
healyhatman 09 Aug, 2019
Here's a fun way to check if it's a double submission! In your submit event, as the first action, use a data builder action to create a {var:} called codestamp, set to {rand:}. In your save data action, save {var:codestamp} to the database.

If it's being SUBMITTED twice, the two codes will be different.
If it's looping incorrectly, the codes will be the same.
bensonley 09 Aug, 2019
@healyhatman ok good idea, will test today.
bensonley 11 Sep, 2019
Answer
Ok after a rather extensive amount of testing I seem to have found the problem, nothing wrong the code, although i have cleaned it up a lot since. It did make me doubt my abilities and logic i have to say!

So... it was down to the class toolbar-button being added to submit buttons which i'd added.

This was causing issues with duplication's on many forms., but never used to so it wasn't actually very easy to spot. I'm still unsure as to why this cause a problem, maybe an update in the semantic js or gecore code? Maybe Max can shed a light on it?

I'm still looking into it and monitoring any developments, but this was arduous and time consuming for something very simple. Hope this helps anyone with the same issues. Thanks for the help also.
This topic is locked and no more replies can be posted.