Jan 29
2014

Magento database connection problems

This post is relevant if you have a Magento database error, likely because you have:

1. Moved your Magento installation to a new host

2. Changed the username and/or password to the database

You will see a problem in /var/reports/xxxxx saying something about “Access denied for user” and “Zend_Db_Adapter_Pdo_Abstract->_connect()”

Here is what to do:

1. Change app/etc/config.xml  to reflect your host, databasename, username and password

2. Change app/etc/local.xml, with the information from step 1.

3. Clear your cache, remove all files in /var/cache

And you are ready to go. Omitting step 2 and 3, which you are likely to do if it is the first time you try this, will normally not fix the problem.

 

Posted in database, magento, webshop | Tagged , , , | Leave a comment
Dec 15
2013

UFW- Ultra simple firewall not that simple?

Recently i experienced a lot of “UFW block” entries in my ufw log in /var/log/ufw.log, unfortunately some few requests to port 80 that were fully allowed, but blocked anyway.

That is not very smart, but finally i discover that when i add a rule that allows a specific port, then ufw adds some extra stuff behind the scenes called “rate limit”, thats the problem here, as we are having quite much load from each client.

The solution is quite simple:

Edit the file /usr/share/pyshared/ufw/backend_iptables.py

Change:

limit_args = ‘-m limit –limit 3/min –limit-burst 10

To:

limit_args = ‘-m limit –limit 100/min –limit-burst 1000

Unfortunately there are no configurable options for this, but for heavy load systems it necessary, or random blocks will occur. Maybe its time to skip ufw, and use iptables directly instead, just thought ufw was so easy, but adding this addional stuff makes it more complicated.

Posted in firewall, linux | Leave a comment
Aug 26
2013

Scannet Webshop 2 – how to make changes to non-editable pages

In Scannet Webshop 2 you can do a lot of stuff, if you have purchased the module “Avanceret design”. However not all pages are available for editing. For example the customers personal page after login is hidden, and it is not possible to change it.

However, using javascript its possible to do some magic with even static pages, that are not available in the Scannet Webshop 2 design system.

In this example we want to disable the feature, where a customer can cancel an order that he has already made. This is a link on the customers personal page, that he sees right after login.

Below i describe what you can do, to disable this link, and you can ofcourse use this technique for almost any layout problem you might have in Scannet Webshop 2.

1. Identify the the tag you need to modify. Open the page in a browser, show the source, and find it, in our case it looks like this:

<a id=”ctl00_bc_hlCancelOrder” href=”/dk/account/orderlist/?link=cancel&amp;returnpath=%2faccount%2fprofile%2f“>Annuller order</a> 

We find that the id of the link we want to modify is: ctl00_bc_hlCancelOrder

2. Go to “Avanceret design”->”Your active template”->Javascript, and you will see the custom javascript that is included on all pages in your shop.

3. Add this function to the code:

function disableCancelOrderLink() {
    var cancelLink = document.getElementById(‘ctl00_bc_hlCancelOrder’);
    if (cancelLink) {
            cancelLink.href=’javascript:alert(\’Not allowed \’)’;
    }  
}

4. And call the function in the document ready handler. Just add this line:

disableCancelOrderLink();

Somewhere in the readyhandler ($(document).ready…)

Thats it, the link is now disabled.

You can do a lot of other tricks, and even manipulate the dom, so you can remove links entirely or inject new structures etc.

I would rather prefer that all pages were accessible from the “Avanceret design” system, they are unfortunately not, but with the above javascript trick its possible, but a bit cumbersome.

Posted in javascript, scannet, webshop | Leave a comment
Aug 07
2013

Linux tips & tricks

This post will be extended over time, and contains small fifs about linux commands that are almost impossible to remember, but nice to have at hand.

Most commands are general and will work on most distributions.

Convert a text file from one character encoding to another:

iconv -f UTF-8 -t ISO-8859-1 textfileutf8.txt > txtfileiso88591.txt

Find top 10 largest files/directories on the system

du -a / | sort -n -r | head -n 10

Replace / with any directory you might need to search in.

Exclude package from being updated by apt-get upgrade

sudo apt-mark hold <package>

 

Posted in linux | Leave a comment
Jul 19
2013

Mysql replication fail

I got this error from the slave in a Mysql replication setup:

Got fatal error 1236: ‘Client requested master to start replication from impossible position’ from master when reading data from binary log’

It happened after a server crash on the master.

So now what? If your information is not very important, you can point the slave to the next bin log file and the replication will continue. You will loose data doing this!

If you cannot loose a single bit of your data, better dump the master database and read it into your slave database.

Do this:

  1. Log in to the slave database
  2. Do a “show slave status”, and find the current mysql-bin file. Increment the number by 1.
  3. STOP SLAVE;
    CHANGE MASTER TO MASTER_LOG_POS = 0;
    CHANGE MASTER TO MASTER_LOG_FILE = 'mysql-bin.00114';
    START SLAVE;
  4. Replace mysql-bin.00114 with the file you found in step 2.
  5. And you are up and running again.

 

Posted in database | Leave a comment
May 22
2013

Orders from Scannet Webshop 2 to e-conomic

A customer needed to move orders from Scannet webshop 2 to e-conomic. Its possible to rent an integrated solution, but it has a number of flaws that makes it unusable for at least this customer, they are:

  • Orders are exported to e-conomic as invoices, that makes the stock function in e-conomic useless and part-delivery in e-conomic is also impossible.
  • Package products (in Danish “pakkeprodukter”) are not handled. You get a wrong stock count on the sub products in a package in e-conomic, as these are not handled.

What did we do? We made our own solution.

We integrated to the webservice at Scannet, to extract orders when they arrive, see more here http://www.scannet.dk/webshop/integration/integration-med-api/

Then we made an integration to the webservice at e-conomic http://www.e-conomic.com/developer

All orders are moved  by a script from the first webservice to the second, called on a regular basis, as a cron job.

To handle package products correct we need a special construction. For each order containing package products we add two orders to e-conomic: One order with all order lines including package products, and a second order with all sub-products belonging to package product with the correct number of pieces, but with zero as orderline price. In this way the stock in e-conomic is counted as it should, and part delivery (on the main order) is possible. The only thing that needs to be done to keep stock updated is to post the second order in the accounts.

Next task is to import electronic orders directly, that is not possible to do in Scannet Webshop 2, the API is read-only, so we need to import directly into e-conomic. Read more about this here https://stokersoft.com/developer/edi/exchange-edi-documents-for-free/

During the process we got a deep knowledge of the api functions/workarounds necessary in both Scannet Webshop 2 and e-conomic, so do not hesitate to contact us at info@stokersoft.com, if you need any assistance in a similar solution.

Posted in economic, edi, webshop | Leave a comment
May 22
2013

Exchange EDI/EDIFACT D96A documents for free

A customer had a problem, they needed to read and generate EDIFACT D96A documents for orders and invoices. They had absolutely nothing, so they were starting from scratch.

The solution is quite simple, and solved with pure and proven open source technology.

  1. Install python 2.7 from http://www.python.org/getit/
  2. Install bots from http://sourceforge.net/projects/bots/
  3. Start the bots-webserver (in Windows its in the bots programs menu)
  4. Login at http://localhost:8080, username: bots password: botsbots
  5. Get the edifact2xml_orders-desadv-invoic_20130123.zip plugin and install it from Systasks->Readplugin
  6. In the web admin setup two channels, one incoming and one outgoing with appropriate folders/ftp accounts, add a route  and choose edifact and write edifact in from messagetype also choose translate for outgoing channel.
  7. Add a test edifact document to the incoming folder and press Run->Run (only new), not check the out folder, it should contain a nice readable XML order.

We had one problem, as the customer exceeded the 35 character limit for the NAD field. Its easy fixed locate c:\Python27\Lib\site-packages\bots\usersys\grammars\edifact\recordsD96AUN.py and extend the limit so it fits.

We need to read the document further into the economic system, but thats quite easy handled now we have the order ready in XML.

If you want to reference an XML stylesheet (xslt) to the XML file, thats easy done. Locate the file c:\Python27\Lib\site-packages\bots\usersys\grammars\edifact\orderes.py

Now change the syntax section, so it looks like this:

syntax = {
'indented': True,
'processing_instructions': [('xml-stylesheet' ,'href="translateorder.xsl" type="text/xml"')]
}

Rename the translateorder.xsl to the name of your xlst transformation file.

Posted in edi | Leave a comment
May 21
2013

A bit of Mysql

Just a few simple things that can be done in mysql, mostly so i have a place i can remember:

Innodb – remove the binlogs

The bin logs in mysql can take up extremely much disk space. Remember to cleanup now and then, e.g. from a cron job, here we keep the last 7 days and delete the rest:

PURGE BINARY LOGS BEFORE DATE_SUB( NOW( ), INTERVAL 7 DAY);

Kill system draining slow queries

If you worked just a bit with mysql you probably tried to issue a query that drained the system completely for ressources. Log in and do this, to stop it:

show processlist;
kill <pid>;

Replace <pid> with the appropriate process id.

From slow to fast query

Always remember to check your queries with the “explain” keyword. It returns the query execution plan, where you can see what if any indexes are used. If you work with larger dataset, do yourself a favor and check your queries that indexes are in place, and that queries are constructed in the right way to use the indexes.

If you cannot predict what queries might be hurting your system, add the following line to my.ini:

long_query_time  = 3
log-slow-queries = [path to the log file]

The above lines in my.ini/my.cnf logs all queries that are taking up more than 3 seconds.

Eventually combine with log-queries-not-using-indexes

Ofcourse you should look deeper if you want to really tune your system, e.g. look at innodb_buffer_pool_size and a lot of other important parameters for innodb users.

Posted in database | Leave a comment
May 20
2013

Reliable and rock solid Java HID USB communication

I did not manage to find a reliable and fully working HID library for Java, so i modified an existing library with focus on stability for running 24/7 Java HID services. Windows only.

The Java HID library is based on the 2011-12-21 version, downloadable from:
http://code.google.com/p/javahidapi/

The modified Java HID library below is running in over 100 production installations 24/7 at the moment, and is proven to be rock stable.

Get the modified and rock stable Java HID library here:
http://www.stokersoft.com/download/javahidapi-20111221_stokersoftfix.zip

If you just want the binary, you can grab it from the zip archive, all source is provided, if you like to compile yourself or inspect the code. Compile with Visual C++ 2010 Express.

The following changes have been made to the original code:

  • Multiple identical devices allowed. The original library does not accept more than one device with same PID/VID, this library supports “unlimited” similar devices.
  • Library now supports multithreading. The original code locks if you access a device from multiple threads.
  • Stability. The old library crashed (JVM crash) very often, and was not suitable for 24/7 use. All code is reviewed for stability and fixed accordingly where necessary. This modified Java HID library is now rock stable. 100 24/7 installations for more than 1,5 year does not lie.

Read about installation/use here http://code.google.com/p/javahidapi/ the functionality has not been changed in this new Java HID library.

Posted in java, usb | Leave a comment
May 20
2013

Extensive logging with Apache Flume

Flume is an interesting distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data.

We plan to use it for a larger setup with multiple heavy traffic servers, so i make a short note here.

Find extensive information about it here:

http://flume.apache.org

 

Posted in logging | Leave a comment