Multi-language Site Consistency Issue

Discussions about development of VICIDIAL and astGUIclient

Moderators: gerski, enjay, williamconley, Op3r, Staydog, gardo, mflorell, MJCoate, mcargile, Kumba, Michael_N

Postby mflorell » Wed Jan 13, 2010 10:18 am

Yes, I do plan on merging your code in a couple weeks, and I don't think there are any major changes planned for the agent interface right now that would cause you to need to re-integrate it again.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Multi-language Site Consistency Issue

Postby caged_penguin » Fri Aug 09, 2013 10:18 am

Hello Matt,

I'm resurrecting this thread since i18n has become a big thing that really needs to be a priority for Vicidial.

The hackish build_translations isn't good.

I too have done the research on using gettext and it won't add any overhead.

I would happily assist in going through and changing to this in the code where

Code: Select all
echo "<some html here>"._("text here")."<close that html";


But to get people using /translations/lang_LOCALE/LC_MESSAGES/messages.po

use that for putting all the translations into and then recompiling/building against a message.mo ..

This will speed up localization efforts greatly.

So can we please re-visit this and using gettext?

Thanks much!

Best Regards,
Danny Williams
Open Source Engineer
https://github.com/mmpreply
Best Regards,
Danny Williams
Open Source Engineer
caged_penguin
 
Posts: 36
Joined: Sun Sep 07, 2008 5:07 pm
Location: florida

Re: Multi-language Site Consistency Issue

Postby williamconley » Sat Aug 10, 2013 2:07 pm

why gettext, why not just a table for text?

after all, the admin pages have very little impact. the only one with ajax is realtime and that's still got very little "language" in it.

and the agent page, ajax, is only called once and after that just data is sent each time, so the language pull is very minimal.

then the editing of languages would be mysql based and easily edited on a simple php web page. all languages live at one time in a single table.

each item has a key and language and it would be loaded onto each page using the language and key to pull from the DB for all the text.
Vicidial Installation and Repair, plus Hosting and Colocation
Newest Product: Vicidial Agent Only Beep - Beta
http://www.PoundTeam.com # 352-269-0000 # +44(203) 769-2294
williamconley
 
Posts: 20018
Joined: Wed Oct 31, 2007 4:17 pm
Location: Davenport, FL (By Disney!)

Re: Multi-language Site Consistency Issue

Postby mflorell » Sun Aug 11, 2013 7:31 am

We've been focusing on security and future compatibility issues with the web interfaces lately, and we should be done with those soon.

After those are done, we have already been approached by a company that has implemented a dynamic text lookup process in the agent interface. This company uses MySQL to store the phrases, which given Vicidial's construction, would probably be the way we would like to go instead of sticking with static text files. This would allow editing of phrases in a web interface, making for a much more easy-to-evolve translation process. We have even thought of adding easy export and import processes which would allow for easy sharing of translations on the vicidial.org website in some way.

So, in closing, it is something we are thinking about. We are planning on working on this after we have finished with the security and future compatibility changes we have been working on for the last several months. It will probably be a MySQL-based approach to storing the phrases that we end up using . And, we will probably start with only the agent interface using this method to start.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Multi-language Site Consistency Issue

Postby williamconley » Sun Aug 11, 2013 8:33 pm

We've done this a couple times with varying results. But the best ones were always MySQL based. (Actually, the best one I remember was MSSQL based, LOL).

That one had an interface which allowed choosing the translation pages by Language, "section" and then by key. The key/language was what actually gets pulled by the Vicidial interface. The Section was merely a method of segmentation to allow users doing translation to locate what they are translating. Imagine trying to change a phrase that appears 200 times on different web pages ... if you can't "look it up" by web page. Worked pretty well. Very easy to add a new language: we just added the language and selected the Code for it, then started adding entries.

I think there was a "default for blank" method that basically stated "if there's no text for a key in this language, grab from language X instead". So a new language goes online instantly with blanks filled in by a Mother language until it's "done".

Of course, this was only for standard left to right languages. And I'm pretty sure Vicidial is in use in a much more Robust range of languages. :)
Vicidial Installation and Repair, plus Hosting and Colocation
Newest Product: Vicidial Agent Only Beep - Beta
http://www.PoundTeam.com # 352-269-0000 # +44(203) 769-2294
williamconley
 
Posts: 20018
Joined: Wed Oct 31, 2007 4:17 pm
Location: Davenport, FL (By Disney!)

Re: Multi-language Site Consistency Issue

Postby caged_penguin » Mon Aug 12, 2013 8:37 am

Code: Select all
[root@localhost html]# cat index.php
<html>
<?php
error_reporting(E_ALL);
$language = "es_ES";
putenv("LANGUAGE=$language");
setlocale(LC_ALL, $language);

$domain = 'messages';
bindtextdomain("$domain", "./locale");
textdomain("$domain");

echo "<td>"._("hello world")."</td>";
echo "<td>"._("this is one")."</td>";
?>
</html>


Code: Select all
[root@localhost html]# php index.php
<html>
<td>ohla mundo</td><td>uno es aqui</td></html>


The reason I feel gettext is the answer is, it really is the less obtrusive to what you already have in place.

All you're doing is wrapping your echo's in _("text here");
Best Regards,
Danny Williams
Open Source Engineer
caged_penguin
 
Posts: 36
Joined: Sun Sep 07, 2008 5:07 pm
Location: florida

Re: Multi-language Site Consistency Issue

Postby williamconley » Mon Aug 12, 2013 1:55 pm

any "rework" would be best done targeting the best solution.

reworking it to create any new method of language translation will be a large project. so ... "quick fix" is out of the question. as such, The Vicidial Group has the ultimate say in what translation method/package is used in Vicidial. Also: Unless you are prepared to Pay for it to be a specific way or perform the translation conversion yourself, it'll go the way Matt says. I'm sure he could be persuaded with $$ or someone else performing the work for free, but not much else.

unless you can make an argument for why your version would be superior to a mysql solution, which would be the only other possibility.
Vicidial Installation and Repair, plus Hosting and Colocation
Newest Product: Vicidial Agent Only Beep - Beta
http://www.PoundTeam.com # 352-269-0000 # +44(203) 769-2294
williamconley
 
Posts: 20018
Joined: Wed Oct 31, 2007 4:17 pm
Location: Davenport, FL (By Disney!)

Re: Multi-language Site Consistency Issue

Postby caged_penguin » Mon Aug 12, 2013 2:02 pm

Hey William,

Considering this was in 2010 and a person had done most of the work to setup gettext() ..

"Postby mflorell » Wed Jan 13, 2010 10:18 am
Yes, I do plan on merging your code in a couple weeks, and I don't think there are any major changes planned for the agent interface right now that would cause you to need to re-integrate it again."

Just sayin :p
Best Regards,
Danny Williams
Open Source Engineer
caged_penguin
 
Posts: 36
Joined: Sun Sep 07, 2008 5:07 pm
Location: florida

Re: Multi-language Site Consistency Issue

Postby williamconley » Mon Aug 12, 2013 9:02 pm

Yep. But times change. I was on them back then to use a mysql solution. Apparently someone else was more convincing than I was. Whatever the cause, I'm just happy that's the way they're going.

It could be pushed aside and change again, or someone could pay for a specific inclusion that has an effect. Or someone could volunteer and do ALL the work so it can just be "included".

Only time will tell.
Vicidial Installation and Repair, plus Hosting and Colocation
Newest Product: Vicidial Agent Only Beep - Beta
http://www.PoundTeam.com # 352-269-0000 # +44(203) 769-2294
williamconley
 
Posts: 20018
Joined: Wed Oct 31, 2007 4:17 pm
Location: Davenport, FL (By Disney!)

Re: Multi-language Site Consistency Issue

Postby DomeDan » Tue Aug 13, 2013 5:53 am

gettext seams cool!
I think I prefer that to the database method

The problem is that its a lot of code editing, but that's the same with the database method.

And we got the regexp method I made http://www.vicidial.org/VICIDIALforum/v ... =3&t=26801 witch is a improvement of the current method

for example this row: www/agc/vicidial.php:7232
var man_status = "Preview the Lead then <a href=\"#\" onclick=\"ManualDialOnly('" + manual_dial_only_type_flag + "')\"><font class=\"preview_text\"> DIAL LEAD</font></a> or <a href=\"#\" onclick=\"ManualDialSkip()\"><font class=\"preview_text\">SKIP LEAD</font></a>";

can be translated with this row:
Preview the Lead then (.*) DIAL LEAD(.*) or (.*)SKIP LEAD|Förhandsgranska Lead och $1 RING LEAD$2 eller $3SKIPPA LEAD|

and btw, this is not theory, it is a working method! http://www.vicidial.org/VICIDIALmantis/view.php?id=663
The only thing that needs to be done is to create/edit the current translation files, no code editing.


Lets make a list of pros and cons!
Code: Select all
           regexp method
    Pros         |     Cons
No code editing  | multiple agc folders
less load        | easier to make mistakes
                 | harder to edit translations

Code: Select all
         database method
    Pros         |     Cons
Friendly editing | code editing
one agc folder   | database load
                 | probably much more code then gettext

Code: Select all
          gettext method
    Pros         |     Cons
one agc folder   | code editing
                 | edit translations?
Vicidial Partner. Region: Sweden/Norway.
Does Vicidial installation, configuration, customization, add-ons, CRM implementation, support, upgrading, network-related, pentesting etc. Remote and onsite assistance.
Email: domedan (at) gmail.com
DomeDan
 
Posts: 1226
Joined: Tue Jan 04, 2011 9:17 am
Location: Sweden

Re: Multi-language Site Consistency Issue

Postby mflorell » Thu Dec 04, 2014 4:41 pm

I just wanted to update everyone in this thread on how our multi-language development this year has been going.

All web interface echoing of text should now be changed to functions using _QXZ("English text"). For right now it will simply echo whatever is in there, but the hard work of going through tens of thousands of lines of PHP code is done. What we just committed to svn/trunk today was the Administrative web interface for managing different languages. To enable languages, you need to go to Admin -> System Settings in the admin.php web interface, then set "Enable Languages" to "1" and submit. Then go to your user modify page and set "Modify Languages" to "1" and submit. Now you will be able to go to Admin -> Languages and you can create new languages, copy existing languages, import languages and export languages.(The import and export processes use the existing "<english>|<other-language>||" file format for UTF-8 text that is used for the language translations files currently in the codebase.)

The last two steps in our multi-language development are: To create a flexible way of assigning languages to agents and/or user groups, and creating an efficient way of looking up the phrases to match the selected language on the fly as the dynamic agent and admin web pages are loaded by users.


Example PHP Code:

echo _QXZ("Closer In Group Choice %1s has been registered to user %2s",0,'',$closer_choice,$user)."\n";

The function call options in order are: "English text", "fixed legth", "l=left or r=right or blank for left", <list of up to 9 PHP variables>

The QXZ function allows for up to 9 defined and ordered variables to be placed in the output using a "Percent-sign, digit, lower-case-s" format (i.e. "%1s") where the digit is the place that the variable is defined at the end of the QXZ function call.



There are also utilities in the "bin" directory that can help you:


/usr/share/astguiclient/ADMIN_www_languages.pl

ADMIN_www_languages.pl - analyze all php scripts for QXZ functions and contents and insert in DB

This script is designed to scan all scripts in the agc and vicidial web directories and gather all of the QXZ phrases present, then insert them into the "www_phrases" database table. After that is done, those phrases can be populated into the "vicidial_language_phrases" database table when adding new Languages in the Administration Web Interface.

/usr/share/astguiclient/ADMIN_www_languages.pl --help

options:
[--help] = this help screen
[--debug] = verbose debug messages
[--debugX] = extra verbose debug messages
[--agc-only] = only parse the agc directory
[--vicidial-only] = only parse the vicidial directory
[--QXZlines] = print full lines that include QXZ functions
[--QXZvalues] = print only QXZ function values
[--QXZvaronly] = print only QXZ function values with PHP variables in them
[--QXZlengthonly] = print only QXZ function values that include a length spec
[--QXZplaceonly] = print only QXZ function values with placeholder variables in them
[--conffile=/path/from/root] = define astguiclient.conf config file to use

Sample output:

FILE PARSING FINISHED!
Files parsed: 283
ConfFile: /etc/astguiclient.conf
QXZ line count: 17957
QXZ count: 24578
QXZ with length set: 1392
QXZ with PHP vars: 681
QXZ with var place set: 179
QXZ with PHP vars inst: 814
QXZ with var place inst: 292

QXZ DB inserts: 6284
QXZ DB dups: 18294



/usr/share/astguiclient/QXZanalyzer.pl --debug --QXZvalues --QXZvaronly --file=/srv/www/htdocs/agc/astguiclient.php

There is a utility stored in the /extras/ directory of the source tree called "QXZanalyzer.pl". This perl script can analyze the PHP scripts one at a time and print out various different types of output to help gather the phrases necessary for future translations as well as to help debug any issues with the PHP code itself

/usr/share/astguiclient/QXZanalyzer.pl --help
QXZanalyzer.pl - analyze php scripts for QXZ functions and contents

options:
[--help] = this help screen
[--debug] = verbose debug messages
[--QXZlines] = print full lines that include QXZ functions
[--QXZvalues] = print only QXZ function values
[--QXZvaronly] = print only QXZ function values with PHP variables in them
[--QXZlengthonly] = print only QXZ function values that include a length spec
[--QXZplaceonly] = print only QXZ function values with placeholder variables in them
[--file=/path/from/root] = define PHP file to analyze

Sample output:

ANALYSIS FINISHED!
File path: /srv/www/htdocs/agc/manager_send.php
Lines in file: 2256
QXZ line count: 130
QXZ count: 130
QXZ with length set: 101
QXZ with PHP vars: 101
QXZ with var place set: 101
QXZ with PHP vars inst: 149
QXZ with var place inst: 149
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Multi-language Site Consistency Issue

Postby mflorell » Fri Dec 05, 2014 6:50 am

Just added the first language export available for download(it's also in svn/trunk):

http://vicidial.org/translations/LANGUA ... 063338.txt

We will be using the following directory to post new languages as they are finished:

http://vicidial.org/translations/
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Multi-language Site Consistency Issue

Postby DomeDan » Mon Dec 08, 2014 9:29 am

Cool! :D I've been looking forward to see progress in multi-language!
This is gonna be a fun upgrade of my system!

Thank you!
Last edited by DomeDan on Tue Dec 09, 2014 6:32 am, edited 1 time in total.
Vicidial Partner. Region: Sweden/Norway.
Does Vicidial installation, configuration, customization, add-ons, CRM implementation, support, upgrading, network-related, pentesting etc. Remote and onsite assistance.
Email: domedan (at) gmail.com
DomeDan
 
Posts: 1226
Joined: Tue Jan 04, 2011 9:17 am
Location: Sweden

Re: Multi-language Site Consistency Issue

Postby mflorell » Mon Dec 08, 2014 10:35 am

Keep in mind we still have a couple pieces left to finish to get the interfaces to use dynamic language display. But, the bulk of the work in changing the coding and the administration of languages is done, and that was the hardest part. As of right now, people CAN start working on translations for when the dynamic translation portion is completed.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Multi-language Site Consistency Issue

Postby mflorell » Wed Dec 17, 2014 7:05 am

As of 2014-12-17(svn revision 2222), languages now follow User Group permissions for agents and managers just like other elements within Vicidial, and they can be assigned to users through the User Modify page, and if the agent has been given permission to change their own language, then the Agent Login screen will show a Language pull-down menu that will allow the agent to change their agent screen language upon logging in to the agent screen. Also, you can now quickly disable languages immediately systemwide by using the settings in the System Settings("Enable Languages" and "Language Method"). The Agent interface is now fully on-the-fly language capable. This means that you can create a language, and add phrases to it, and as you make changes to the language phrases, they are available immediately to the agent screen to display.

The last step is: To get on-the-fly translation working for all admin web pages.

We have also put a new Spanish language file on the translations web folder(it has some new phrases and fixes several issues with the old one):

http://vicidial.org/translations/LANGUA ... 224232.txt
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Multi-language Site Consistency Issue

Postby mflorell » Sun Dec 21, 2014 4:33 pm

New Dutch(nl) agent-only translation has been added:

http://vicidial.org/translations/LANGUA ... 162650.txt
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Multi-language Site Consistency Issue

Postby mflorell » Sun Dec 28, 2014 11:05 am

Added several agent translations(Polish, Portuguese, Russian, Swedish, Japanese, Chinese, Slovak, Danish) and updated the Spanish and Dutch agent dictionaries:

http://vicidial.org/translations/
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Multi-language Site Consistency Issue

Postby mflorell » Mon Dec 29, 2014 3:14 pm

Added the last 5 agent-only language translations(Greek, Italian, German, Brazilian Portuguese and French),

http://vicidial.org/translations/


I also fixed a few things in the agent code, so an svn/trunk update would be recommended at this point.


So, now all 16 languages are available for the agent interface for on-the-fly translations, and the work begins on making the admin interface on-the-fly translation capable. I'll post more updates as the work is completed.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Multi-language Site Consistency Issue

Postby mflorell » Wed Dec 31, 2014 10:21 am

The Admin web pages are all now capable of on-the-fly translation in svn/trunk. I have also added a new "Change language" page in the admin interface, it is linked at the top of the admin.php screen.

I have also updated the Spanish ALL dictionary file(version 004).

The very last phase to complete on this project is finalizing the other 5 languages that have had admin translations in the past(French, German, Italian, Greek and Brazilian Portuguese).
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Multi-language Site Consistency Issue

Postby mflorell » Tue Jan 06, 2015 6:14 pm

All previous admin languages have been completed. This project is finally done!

I look forward to people upgrading their systems and uploading their own language files. The new software makes that so much easier now.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Multi-language Site Consistency Issue

Postby DomeDan » Wed Jan 07, 2015 9:50 am

Nice!
yeah though the QXZ function looked a little incomplete when I upgraded :P time to do an other upgrade!
checked out the phrase translator tool, its great! so simple to look what need to be translated and easy to import new phrases! its awsome!
Vicidial Partner. Region: Sweden/Norway.
Does Vicidial installation, configuration, customization, add-ons, CRM implementation, support, upgrading, network-related, pentesting etc. Remote and onsite assistance.
Email: domedan (at) gmail.com
DomeDan
 
Posts: 1226
Joined: Tue Jan 04, 2011 9:17 am
Location: Sweden

Re: Multi-language Site Consistency Issue

Postby mflorell » Wed Jan 07, 2015 10:03 am

Thanks DomeDan!

Yes, this has been an active development project for 6 months, it's one of the most time-consuming projects that we have done to date(and VERY tedious too).

I'm glad you like the Language admin interface, and I look forward to seeing you post your first translation export to the Issue Tracker :)
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Multi-language Site Consistency Issue

Postby ccabrera » Mon Jan 12, 2015 8:38 pm

Matt,

I´d like to help with the spanish translation. What would be the best process to follow in order to avoid duplicating translation efforts? I mean, I wouldn´t like to translate something someone else is working on, and I´m thinking abount using some open source projects (example: http://zanata.org) in order to achieve an efficient collaborative environment.

Do you have any suggestions regarding this matter?
Christian Cabrera
Enlaza Comunicaciones - Vicidial Partner
Mexico City
ccabrera
 
Posts: 153
Joined: Fri Jan 14, 2011 7:53 pm
Location: Mexico City

Re: Multi-language Site Consistency Issue

Postby mflorell » Mon Jan 12, 2015 10:25 pm

Thank you ccabrera!

I have created a new Zanata.org project for Vicidial if you would like to try that platform for translations,
https://translate.zanata.org/zanata/pro ... ial_web_01

I don't know of anyone that is currently working on a Spanish translation. The current Spanish translation that we are using is available at this link(also uploaded on zanata.org),
http://vicidial.org/translations/LANGUA ... 101146.txt

You can either work on the Zanata.org website or open a new Issue Tracker ticket to upload translation files that you edit yourself.

If you have any other questions, please let me know.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Previous

Return to Development

Who is online

Users browsing this forum: No registered users and 35 guests