http://159.203.101.162/w/api.php?action=feedcontributions&feedformat=atom&user=Masaru20100 2026-05-17T14:26:16Z User contributions MediaWiki 1.31.1 http://159.203.101.162/w/index.php?title=Pathfinder_PFS_Information&diff=4026 2016-08-25T14:59:15Z

<p>Masaru20100: </p> <hr /> <div>PCGen for Pathfinder Society Organized Play<br /> <br /> This page is meant to include all the information for players that want to use PCGen to track their characters.<br /> <br /> The guide is written with the assumption that you are familiar with PCGen.<br /> <br /> == Sources ==<br /> <br /> When selecting sources, you have two options. You can either use the source that tries to include all the items that are available to characters (this source tries to keep up with the additionnal resources page and new source included in PCGen), or you can manually add sources and only select allowed equipment/feat/races/etc.<br /> <br /> Using the first option, everything is included in the single source Guide To Pathfinder Society Organized Play (GTPSOP). When you load this source, all other sources mentioned in this source and in the Additional Resources Guide are modified according to the rules in GTPSOP.<br /> <br /> == XP ==<br /> <br /> There is a Pathfinder Society mode for XP leveling, but you can’t track ½ (gained through slow advancement) as the XP in PCGen is an integer at the moment.<br /> <br /> == Game Mode ==<br /> <br /> The second thing is that there are two game modes that support PSOP.<br /> <br /> - Pathfinder PFS: This is the game mode designed to eliminate all source books not specifically approved from PSOP or existing sources that are not yet converted to PSOP from the source selection user interface. You can use this game mode if you want to use only using PSOP converted sources that are legal for use in PSOP. It should be noted that while I spent quite a lot of time working on these sources it is possible that some things slipped through. So if you see something that PCGen includes that seems too good to be true, you should check with the real GTPSOP and the Additional Resources Guide to be sure.<br /> <br /> - Pathfinder RPG: This is the normal game mode used if you want to use GTPSOP but also want to be able to use unconverted sources. This is ideal for players that use PSOP rules as a basis for their home games, but choose to also include additional unapproved sources. This game mode can also be used if you want to include additional sources that exist in PCGen and the Additional Resources Guide, but have not yet been converted to PSOP use in PCGen. If you use this game mode for characters being played in PSOP and you add sources that are not yet converted to PFOP by PCGen, you will have to check with the Additional Resources Guide yourself to make sure the abilities and items you have chosen are approved for PSOP use. The list of PSOP sources that have been converted are listed in the info box for the source Guide to Pathfinder Society Organized Play in the advances sources selection dialog.<br /> <br /> <br /> <br /> Now... how to use it:<br /> <br /> <br /> <br /> Pathfinder Society expect players to own all books that contain rules or items that their characters use. The way you do this is to select the Multi-Set called Pathfinder Society Core Assumption (which includes the Core Rulebook and Guide to Pathfinder Society Organized Play and Chronicle Sheets boons and abilities [more on that below]). Then select each book source that you personally own. Load them and then create or load your character using those sources. You can also use the Multi-Set called &quot;Pathfinder Society&quot;. This is the set that includes every PSOP source that PCGen supports. If you use this set in a PSOP game be aware that you will be presented with abilities and items from sources that you personally don't own and if you include them on a character you play in PSOP, this is cheating and a GM can ask you to leave the game or worse.<br /> <br /> <br /> <br /> Now, on to your character sheet... Pathfinder society characters will make frequent use of the Background and Chronicles tabs. So here's how they work:<br /> <br /> <br /> <br /> - Society Faction: This is selected on the Background tab. If you had a character that was originally associated with a faction that is no longer allowed, such as the Shadow Lodge, you add your original faction here. You will then use the chronicle sheet and the boon from that chronicle sheet to allow an additional faction selection.<br /> <br /> <br /> <br /> - Prestige Awards: This is selected on the Background tab. As your character gains fame, you add Fame points to this ability pool and spend the fame points on various prestige awards defined in GTPSOP, Faction Guide and other sources. Unspent fame can be spent on the &quot;Current Prestige Award&quot; ability which will allow you to resolve the reminder and have an ability show on the output sheet that shows the total remaining points available.<br /> <br /> <br /> <br /> - Chronicle Sheets: This is selected on the Chronicles tab. PCGen includes boons and items specified on chronicle sheets from seasons 0-4, many modules and some special event sheets. When your character gains a chronicle sheet form a game or special event, you add the sheet to your character and select the subtier that you played. In the gase of playing a scenario between tiers, you need to select the subtier actually played so the selection of boons and items will be accurate. PCGen does not track XP or GP gained on chronicle sheets. When you add a chronicle sheet that has boons to your character a new ability pool will be shown that allows you to add the boon to your character. Boons for save bonuses, skill bonuses, etc are shown on the fantasy PDF output sheets, the standard htm.ftl output sheet and on the standard preview htm.ftl output sheets. These output sheets also include a section that documents the chronicle sheets you have applied to the character.<br /> <br /> <br /> <br /> OPTIONAL: Under PCGen &gt; Tools &gt; Preferences &gt; House Rules you can select the option DisplayFullAbility. If you do this, the chronicle sheet section of the output sheet will include a list of all purchasable items listed on the standard chronicle sheet according to the subtier you selected. Note that in some scenarios, some items may not actually be available to your character and may have been crossed out by the GM for items that you didn't find. Display of these items if for convenience only and you should always confirm the items on your actual chronicle sheets prior to purchasing the items for your character.<br /> <br /> <br /> <br /> NOTE: While the chronicle sheets applied are documented on the output sheet, the order of the chronicle sheets is not.<br /> <br /> == Inventory Tracking Sheet ==<br /> <br /> Lastly, there is no fully automated method in PCGen that duplicated the inventory tracking sheet required by PSOP. However, there is several workarounds.<br /> <br /> The first one is to track ITS separately on its own sheet (of paper), in a libreoffice sheet, etc.<br /> <br /> Second one uses the ability to customize equipment. When you buy each inventory item you can customize it and alter the name to, for example (#12) to specify on which chronicle sheet you purchased this item. Then to track items sold or used, you would customize a bag or other container items and replace its name with &quot;Sold #12&quot; or whatever and move the items sold for that chronicle into that bag. That bag would go into the equipment Not Carried. Oh, and there is a trick to handling items like wands. Once you use the first charge of a wand, add a new identical wand with the same name adjustment to identify which chronicle sheet it was purchased on and add that into a non-carried bag titled &quot;Ammunition Used (#12)&quot; or whatever and adjust its charges to the number of charges used, while reducing the in-use equipped or carried item by the number of charges used. If you're careful and diligent about selecting the proper buy/sell rates when purchasing and upgrading equipment, and moving used or sold items into the appropriate not-carried containers, PCGen will be able to total the equipment value of all the equipment you have ever purchased for your character.<br /> <br /> Third way is to use the campaign chronicle tracker (in description). For each sheet, you can have an entry where you list what you used/bought. This will be exported (only with the PDF export for now). You could also use a custom description page instead.<br /> <br /> == References ==<br /> <br /> Pathfinder Society Organized Play home page<br /> <br /> Guide to the Pathfinder Society Organized Play (find the link to Paizo)</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Pathfinder_PFS_Information&diff=4025 2016-08-25T11:32:31Z

<p>Masaru20100: Improve page</p> <hr /> <div>PCGen for Pathfinder Society Organized Play<br /> <br /> This page is meant to include all the information for players that want to use PCGen to track their characters.<br /> <br /> The guide is written with the assumption that you are familiar with PCGen.<br /> <br /> == Sources ==<br /> <br /> When selecting sources, you have two options. You can either use the source that tries to include all the items that are available to characters (this source tries to keep up with the additionnal resources page and new source included in PCGen), or you can manually add sources and only select allowed equipment/feat/races/etc.<br /> <br /> Using the first option, everything is included in the single source Guide To Pathfinder Society Organized Play (GTPSOP). When you load this source, all other sources mentioned in this source and in the Additional Resources Guide are modified according to the rules in GTPSOP.<br /> <br /> == Game Mode ==<br /> <br /> The second thing is that there are two game modes that support PSOP.<br /> <br /> - Pathfinder PFS: This is the game mode designed to eliminate all source books not specifically approved from PSOP or existing sources that are not yet converted to PSOP from the source selection user interface. You can use this game mode if you want to use only using PSOP converted sources that are legal for use in PSOP. It should be noted that while I spent quite a lot of time working on these sources it is possible that some things slipped through. So if you see something that PCGen includes that seems too good to be true, you should check with the real GTPSOP and the Additional Resources Guide to be sure.<br /> <br /> - Pathfinder RPG: This is the normal game mode used if you want to use GTPSOP but also want to be able to use unconverted sources. This is ideal for players that use PSOP rules as a basis for their home games, but choose to also include additional unapproved sources. This game mode can also be used if you want to include additional sources that exist in PCGen and the Additional Resources Guide, but have not yet been converted to PSOP use in PCGen. If you use this game mode for characters being played in PSOP and you add sources that are not yet converted to PFOP by PCGen, you will have to check with the Additional Resources Guide yourself to make sure the abilities and items you have chosen are approved for PSOP use. The list of PSOP sources that have been converted are listed in the info box for the source Guide to Pathfinder Society Organized Play in the advances sources selection dialog.<br /> <br /> <br /> <br /> Now... how to use it:<br /> <br /> <br /> <br /> Pathfinder Society expect players to own all books that contain rules or items that their characters use. The way you do this is to select the Multi-Set called Pathfinder Society Core Assumption (which includes the Core Rulebook and Guide to Pathfinder Society Organized Play and Chronicle Sheets boons and abilities [more on that below]). Then select each book source that you personally own. Load them and then create or load your character using those sources. You can also use the Multi-Set called &quot;Pathfinder Society&quot;. This is the set that includes every PSOP source that PCGen supports. If you use this set in a PSOP game be aware that you will be presented with abilities and items from sources that you personally don't own and if you include them on a character you play in PSOP, this is cheating and a GM can ask you to leave the game or worse.<br /> <br /> <br /> <br /> Now, on to your character sheet... Pathfinder society characters will make frequent use of the Background and Chronicles tabs. So here's how they work:<br /> <br /> <br /> <br /> - Society Faction: This is selected on the Background tab. If you had a character that was originally associated with a faction that is no longer allowed, such as the Shadow Lodge, you add your original faction here. You will then use the chronicle sheet and the boon from that chronicle sheet to allow an additional faction selection.<br /> <br /> <br /> <br /> - Prestige Awards: This is selected on the Background tab. As your character gains fame, you add Fame points to this ability pool and spend the fame points on various prestige awards defined in GTPSOP, Faction Guide and other sources. Unspent fame can be spent on the &quot;Current Prestige Award&quot; ability which will allow you to resolve the reminder and have an ability show on the output sheet that shows the total remaining points available.<br /> <br /> <br /> <br /> - Chronicle Sheets: This is selected on the Chronicles tab. PCGen includes boons and items specified on chronicle sheets from seasons 0-4, many modules and some special event sheets. When your character gains a chronicle sheet form a game or special event, you add the sheet to your character and select the subtier that you played. In the gase of playing a scenario between tiers, you need to select the subtier actually played so the selection of boons and items will be accurate. PCGen does not track XP or GP gained on chronicle sheets. When you add a chronicle sheet that has boons to your character a new ability pool will be shown that allows you to add the boon to your character. Boons for save bonuses, skill bonuses, etc are shown on the fantasy PDF output sheets, the standard htm.ftl output sheet and on the standard preview htm.ftl output sheets. These output sheets also include a section that documents the chronicle sheets you have applied to the character.<br /> <br /> <br /> <br /> OPTIONAL: Under PCGen &gt; Tools &gt; Preferences &gt; House Rules you can select the option DisplayFullAbility. If you do this, the chronicle sheet section of the output sheet will include a list of all purchasable items listed on the standard chronicle sheet according to the subtier you selected. Note that in some scenarios, some items may not actually be available to your character and may have been crossed out by the GM for items that you didn't find. Display of these items if for convenience only and you should always confirm the items on your actual chronicle sheets prior to purchasing the items for your character.<br /> <br /> <br /> <br /> NOTE: While the chronicle sheets applied are documented on the output sheet, the order of the chronicle sheets is not.<br /> <br /> == Inventory Tracking Sheet ==<br /> <br /> Lastly, there is no fully automated method in PCGen that duplicated the inventory tracking sheet required by PSOP. However, there is several workarounds.<br /> <br /> The first one is to track ITS separately on its own sheet (of paper), in a libreoffice sheet, etc.<br /> <br /> Second one uses the ability to customize equipment. When you buy each inventory item you can customize it and alter the name to, for example (#12) to specify on which chronicle sheet you purchased this item. Then to track items sold or used, you would customize a bag or other container items and replace its name with &quot;Sold #12&quot; or whatever and move the items sold for that chronicle into that bag. That bag would go into the equipment Not Carried. Oh, and there is a trick to handling items like wands. Once you use the first charge of a wand, add a new identical wand with the same name adjustment to identify which chronicle sheet it was purchased on and add that into a non-carried bag titled &quot;Ammunition Used (#12)&quot; or whatever and adjust its charges to the number of charges used, while reducing the in-use equipped or carried item by the number of charges used. If you're careful and diligent about selecting the proper buy/sell rates when purchasing and upgrading equipment, and moving used or sold items into the appropriate not-carried containers, PCGen will be able to total the equipment value of all the equipment you have ever purchased for your character.<br /> <br /> Third way is to use the campaign chronicle tracker (in description). For each sheet, you can have an entry where you list what you used/bought. This will be exported (only with the PDF export for now). You could also use a custom description page instead.</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Subversion_Setup_(deprecated)&diff=3245 2012-12-12T09:03:01Z

<p>Masaru20100: /* NEW Update with SF Upgrade */ nowiki for svn command</p> <hr /> <div> {| align=&quot;right&quot;<br /> | __TOC__<br /> |}<br /> <br /> ==Introduction==<br /> <br /> Subversion is used as the version control system for the sources of the project — it holds all of the program source code and data that make up PCGen. It allows us to track changes in the various files, share our work with each other and to coordinate our activity in a single location.<br /> <br /> Developers use a subversion client to download the sources (called checkout) and update the sources with changes (called commit). The [http://subversion.apache.org/ official site] propose binaries or links to them for many operating systems. Most GNU/Linux distributions propose it as a package; in Debian it is the &lt;a href=&quot;apt:subversion&gt;subversion package&lt;/a&gt;. One commonly used Windows subversion client, [[#TortoiseSVN|TortoiseSVN]], is described below. Most integrated development environments (IDEs) include Subversion support. In the case of [[Eclipse]], it is by using the the [http://subclipse.tigris.org/servlets/ProjectProcess?pageID=p4wYuA subclipse] plugin.<br /> <br /> =NEW Update with SF Upgrade=<br /> <br /> On Beta projects (versus Classic projects), Sourceforge seems to be encouraging the use of svn+ssh rather than https. It seems that using http is still possible with Beta projects, maybe limited to read only access.<br /> <br /> The command to get the trunk and put it in the pcgen-trunk directory is:<br /> &lt;nowiki&gt;svn checkout https://svn.code.sf.net/p/pcgen/code/Trunk pcgen-trunk&lt;/nowiki&gt;<br /> <br /> ==SVN+SSH Access==<br /> <br /> It is possible to use SSH to get the repository simply with read only access. The command to get the trunk is:<br /> &lt;nowiki&gt;svn checkout svn://svn.code.sf.net/p/pcgen/code/Trunk pcgen-trunk&lt;/nowiki&gt;<br /> <br /> To get a read/write access, the command, where USERNAME has to be changed to your Sourceforge account id is:<br /> &lt;nowiki&gt;svn checkout --username=USERNAME svn+ssh://USERNAME@svn.code.sf.net/p/pcgen/code/Trunk pcgen-trunk&lt;/nowiki&gt;<br /> <br /> It will be asking your Sourceforge password.<br /> <br /> It is also possible to use a SSH key to avoid sending your encrypted password over the network. It is more secure, because to access PCGen repository, they need a file on your computer, not just your password. If the SSH key file is protected by a password, the protection is more improved.<br /> The following paragraphs describe the procedure needed in order to use the key file.<br /> <br /> ===On Windows===<br /> <br /> '''Install PuttyGen'''<br /> <br /> puttygen.exe - http://the.earth.li/~sgtatham/putty/latest/x86/puttygen.exe<br /> <br /> <br /> '''Generate a SSH Key (file ends with .ppk)'''<br /> <br /> * run puttygen<br /> * Generate a New Key - SSH2<br /> * IN the comment section add 'sourceforgeUserId@shell.sourceforge.net'<br /> * OPTIONAL - You may set a passphrase, SF recommends it.<br /> * click the Save private key button<br /> * enter a filename for your new .ppk file when prompted<br /> * press Save<br /> * Copy the Public Key Information<br /> * Go to your SF Developer Account Services<br /> * Edit Keys<br /> * Paste your Public Key here<br /> <br /> <br /> '''Install Pageant'''<br /> <br /> pageant.exe - http://the.earth.li/~sgtatham/putty/latest/x86/pageant.exe<br /> <br /> <br /> '''Run the ageant'''<br /> <br /> * double-click on pageant.exe<br /> * you should now see a computer with a black hat in the tray<br /> * Add your .ppk to the ageant<br /> <br /> * right-click on the computer with the black hat in your taskbar<br /> * select Add Key from the context menu<br /> * browse to your recently-created .ppk file and select it<br /> * click Open<br /> * enter your password when prompted (If you set the Passphrase earlier in this set up for your key, this is what it is asking for)<br /> * verify that your key has been added by double-clicking on the computer with the black hat<br /> <br /> If you don't want to repeat this procedure after every reboot of your client, you should place Pageant in the auto-start group of your Windows installation. You can append the keys with complete paths as command line arguments to Pageant.exe.<br /> <br /> '''Install Plink'''<br /> <br /> plink.exe - http://the.earth.li/~sgtatham/putty/latest/x86/plink.exe<br /> <br /> <br /> '''Install TortoiseSVN'''<br /> <br /> TortoiseSVN download page<br /> <br /> <br /> '''Configure TortoiseSVN'''<br /> <br /> You need to tell TortoiseSVN to use TortoisePlink to handle SSH traffic. To do this see the following steps:<br /> <br /> * right-click on an explorer window somewhere<br /> * hover over TortoiseSVN in the context menu<br /> * select Settings from the sub-menu<br /> * select Network from the list on the left<br /> * under the SSH box in the right side, click the Browse... button<br /> * browse to and select TortoisePlink.exe (mine is in c:\Program Files\TortoiseSVN\bin\TortoisePlink.exe)<br /> * click Open<br /> * click Apply<br /> <br /> ===On GNU/Linux===<br /> <br /> On top of subversion client, an ssh client is needed. OpenSSH is included in most distributions, on Debian, the package is openssh-client.<br /> <br /> If you don’t have a key yet, create one with:<br /> ssh-keygen<br /> It will ask you where to store it (default is fine), and an eventual password. The public and private key will then be generated.<br /> <br /> <br /> This command is used to copy the public key to an SSH server (in the case of the default location to store the key):<br /> ssh-copy-id USERNAME@svn.code.sf.net<br /> <br /> <br /> Then you can use this command to grab the project:<br /> svn checkout --username=USERNAME svn+ssh://USERNAME@svn.code.sf.net/p/pcgen/code/Trunk pcgen-trunk<br /> <br /> In my case, my password to the id key was asked then my Sourceforge account key was asked.<br /> <br /> <br /> To be Done: testing commits with this configuration.<br /> <br /> ===On Mac OS X===<br /> <br /> [http://developer.apple.com/library/mac/#documentation/MacOSXServer/Conceptual/XServer_ProgrammingGuide/Articles/SSH.html Apple Developer SSH Page]<br /> <br /> =Related Articles=<br /> <br /> * Macintosh instructions can be found at [[SVN on the Mac]]<br /> * [[Web access to SVN]]<br /> * [[Merging]] with Subversion<br /> * [[Basic Developer Setup#Setting up with Eclipse|Setting up with Eclipse]]<br /> <br /> = Using SVN =<br /> <br /> The Subversion command line client is not terribly hard to use either, especially if you are familiar with CVS. You can download [http://subversion.tigris.org/project_packages.html binaries] for various platforms, including Windows.<br /> <br /> Note the section on that page that says that the Subversion development team does *not* directly support these binaries. They mean it. However, questions about these posted to the [http://subversion.tigris.org/servlets/ProjectMailingListList Subversion users mailing list] are usually answered pretty quickly.<br /> <br /> <br /> ==TortoiseSVN==<br /> <br /> TSVN is a GUI client for Subversion repositories. Before installing TSVN, please note that it is a Windows Explorer Shell extension, and as such, will not work on any other operating system.<br /> <br /> Make sure you have version 3.0 or later of the Windows Installer. This should be included in Windows Update for both Windows XP and Windows 2000. If for some reason you are still stuck on Windows 2000 (or earlier!), and can't update to the latest Service Pack, you are out of luck. I don't know if TSVN will work on Windows NT or Windows 98.<br /> <br /> # Close down all applications that might be running.<br /> # Download the [http://tortoisesvn.net/downloads TortoiseSVN Program] and the [http://tortoisesvn.sourceforge.net/?q=support manual].<br /> # Run the MSI.<br /> # Restart your computer.<br /> # Make sure you actually restarted your machine, not just logged off. TSVN is a shell extension, which means that you *must* restart your machine after installing.<br /> <br /> After you've installed TSVN, your Windows Explorer right-click context menus will all have a new entry called &quot;Tortoise SVN&quot;. Click this new entry to see the options available. Exactly how you will connect to a repository depends on how SourceForge has Subversion set up. I'll update this entry with more information when it becomes available.<br /> <br /> TSVN uses it's own SSH client, based on PuTTY, so you don't have to have it installed separately. The first time you attempt to browse a repository or do a check out, it will ask you for your credentials. You have the option of saving those credentials for future sessions as well.<br /> <br /> See the TSVN user's [http://tortoisesvn.tigris.org/list_etiquette.html mailing list information page] to subscribe to the mailing list. There are bunches of knowledgeable people there to help with virtually any problem you might have. Please make sure you follow proper nettiquette on this list, and research any questions you might have before posting something as a bug.<br /> <br /> ==RapidSVN==<br /> <br /> If you are using an operating system other than Windows, and you want a GUI client, there is a cross platform GUI client called [http://rapidsvn.tigris.org/ RapidSVN], but I have no experience, good or bad, with that program.<br /> <br /> == Subclipse ==<br /> <br /> There is also an [http://subclipse.tigris.org/ Eclipse plug-in] that is highly thought of, see [[Basic Developer Setup]] for details.</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Subversion_Setup_(deprecated)&diff=3244 2012-12-12T09:02:11Z

<p>Masaru20100: /* SVN+SSH Access */</p> <hr /> <div> {| align=&quot;right&quot;<br /> | __TOC__<br /> |}<br /> <br /> ==Introduction==<br /> <br /> Subversion is used as the version control system for the sources of the project — it holds all of the program source code and data that make up PCGen. It allows us to track changes in the various files, share our work with each other and to coordinate our activity in a single location.<br /> <br /> Developers use a subversion client to download the sources (called checkout) and update the sources with changes (called commit). The [http://subversion.apache.org/ official site] propose binaries or links to them for many operating systems. Most GNU/Linux distributions propose it as a package; in Debian it is the &lt;a href=&quot;apt:subversion&gt;subversion package&lt;/a&gt;. One commonly used Windows subversion client, [[#TortoiseSVN|TortoiseSVN]], is described below. Most integrated development environments (IDEs) include Subversion support. In the case of [[Eclipse]], it is by using the the [http://subclipse.tigris.org/servlets/ProjectProcess?pageID=p4wYuA subclipse] plugin.<br /> <br /> =NEW Update with SF Upgrade=<br /> <br /> On Beta projects (versus Classic projects), Sourceforge seems to be encouraging the use of svn+ssh rather than https. It seems that using http is still possible with Beta projects, maybe limited to read only access.<br /> <br /> The command to get the trunk and put it in the pcgen-trunk directory is:<br /> svn checkout https://svn.code.sf.net/p/pcgen/code/Trunk pcgen-trunk<br /> <br /> ==SVN+SSH Access==<br /> <br /> It is possible to use SSH to get the repository simply with read only access. The command to get the trunk is:<br /> &lt;nowiki&gt;svn checkout svn://svn.code.sf.net/p/pcgen/code/Trunk pcgen-trunk&lt;/nowiki&gt;<br /> <br /> To get a read/write access, the command, where USERNAME has to be changed to your Sourceforge account id is:<br /> &lt;nowiki&gt;svn checkout --username=USERNAME svn+ssh://USERNAME@svn.code.sf.net/p/pcgen/code/Trunk pcgen-trunk&lt;/nowiki&gt;<br /> <br /> It will be asking your Sourceforge password.<br /> <br /> It is also possible to use a SSH key to avoid sending your encrypted password over the network. It is more secure, because to access PCGen repository, they need a file on your computer, not just your password. If the SSH key file is protected by a password, the protection is more improved.<br /> The following paragraphs describe the procedure needed in order to use the key file.<br /> <br /> ===On Windows===<br /> <br /> '''Install PuttyGen'''<br /> <br /> puttygen.exe - http://the.earth.li/~sgtatham/putty/latest/x86/puttygen.exe<br /> <br /> <br /> '''Generate a SSH Key (file ends with .ppk)'''<br /> <br /> * run puttygen<br /> * Generate a New Key - SSH2<br /> * IN the comment section add 'sourceforgeUserId@shell.sourceforge.net'<br /> * OPTIONAL - You may set a passphrase, SF recommends it.<br /> * click the Save private key button<br /> * enter a filename for your new .ppk file when prompted<br /> * press Save<br /> * Copy the Public Key Information<br /> * Go to your SF Developer Account Services<br /> * Edit Keys<br /> * Paste your Public Key here<br /> <br /> <br /> '''Install Pageant'''<br /> <br /> pageant.exe - http://the.earth.li/~sgtatham/putty/latest/x86/pageant.exe<br /> <br /> <br /> '''Run the ageant'''<br /> <br /> * double-click on pageant.exe<br /> * you should now see a computer with a black hat in the tray<br /> * Add your .ppk to the ageant<br /> <br /> * right-click on the computer with the black hat in your taskbar<br /> * select Add Key from the context menu<br /> * browse to your recently-created .ppk file and select it<br /> * click Open<br /> * enter your password when prompted (If you set the Passphrase earlier in this set up for your key, this is what it is asking for)<br /> * verify that your key has been added by double-clicking on the computer with the black hat<br /> <br /> If you don't want to repeat this procedure after every reboot of your client, you should place Pageant in the auto-start group of your Windows installation. You can append the keys with complete paths as command line arguments to Pageant.exe.<br /> <br /> '''Install Plink'''<br /> <br /> plink.exe - http://the.earth.li/~sgtatham/putty/latest/x86/plink.exe<br /> <br /> <br /> '''Install TortoiseSVN'''<br /> <br /> TortoiseSVN download page<br /> <br /> <br /> '''Configure TortoiseSVN'''<br /> <br /> You need to tell TortoiseSVN to use TortoisePlink to handle SSH traffic. To do this see the following steps:<br /> <br /> * right-click on an explorer window somewhere<br /> * hover over TortoiseSVN in the context menu<br /> * select Settings from the sub-menu<br /> * select Network from the list on the left<br /> * under the SSH box in the right side, click the Browse... button<br /> * browse to and select TortoisePlink.exe (mine is in c:\Program Files\TortoiseSVN\bin\TortoisePlink.exe)<br /> * click Open<br /> * click Apply<br /> <br /> ===On GNU/Linux===<br /> <br /> On top of subversion client, an ssh client is needed. OpenSSH is included in most distributions, on Debian, the package is openssh-client.<br /> <br /> If you don’t have a key yet, create one with:<br /> ssh-keygen<br /> It will ask you where to store it (default is fine), and an eventual password. The public and private key will then be generated.<br /> <br /> <br /> This command is used to copy the public key to an SSH server (in the case of the default location to store the key):<br /> ssh-copy-id USERNAME@svn.code.sf.net<br /> <br /> <br /> Then you can use this command to grab the project:<br /> svn checkout --username=USERNAME svn+ssh://USERNAME@svn.code.sf.net/p/pcgen/code/Trunk pcgen-trunk<br /> <br /> In my case, my password to the id key was asked then my Sourceforge account key was asked.<br /> <br /> <br /> To be Done: testing commits with this configuration.<br /> <br /> ===On Mac OS X===<br /> <br /> [http://developer.apple.com/library/mac/#documentation/MacOSXServer/Conceptual/XServer_ProgrammingGuide/Articles/SSH.html Apple Developer SSH Page]<br /> <br /> =Related Articles=<br /> <br /> * Macintosh instructions can be found at [[SVN on the Mac]]<br /> * [[Web access to SVN]]<br /> * [[Merging]] with Subversion<br /> * [[Basic Developer Setup#Setting up with Eclipse|Setting up with Eclipse]]<br /> <br /> = Using SVN =<br /> <br /> The Subversion command line client is not terribly hard to use either, especially if you are familiar with CVS. You can download [http://subversion.tigris.org/project_packages.html binaries] for various platforms, including Windows.<br /> <br /> Note the section on that page that says that the Subversion development team does *not* directly support these binaries. They mean it. However, questions about these posted to the [http://subversion.tigris.org/servlets/ProjectMailingListList Subversion users mailing list] are usually answered pretty quickly.<br /> <br /> <br /> ==TortoiseSVN==<br /> <br /> TSVN is a GUI client for Subversion repositories. Before installing TSVN, please note that it is a Windows Explorer Shell extension, and as such, will not work on any other operating system.<br /> <br /> Make sure you have version 3.0 or later of the Windows Installer. This should be included in Windows Update for both Windows XP and Windows 2000. If for some reason you are still stuck on Windows 2000 (or earlier!), and can't update to the latest Service Pack, you are out of luck. I don't know if TSVN will work on Windows NT or Windows 98.<br /> <br /> # Close down all applications that might be running.<br /> # Download the [http://tortoisesvn.net/downloads TortoiseSVN Program] and the [http://tortoisesvn.sourceforge.net/?q=support manual].<br /> # Run the MSI.<br /> # Restart your computer.<br /> # Make sure you actually restarted your machine, not just logged off. TSVN is a shell extension, which means that you *must* restart your machine after installing.<br /> <br /> After you've installed TSVN, your Windows Explorer right-click context menus will all have a new entry called &quot;Tortoise SVN&quot;. Click this new entry to see the options available. Exactly how you will connect to a repository depends on how SourceForge has Subversion set up. I'll update this entry with more information when it becomes available.<br /> <br /> TSVN uses it's own SSH client, based on PuTTY, so you don't have to have it installed separately. The first time you attempt to browse a repository or do a check out, it will ask you for your credentials. You have the option of saving those credentials for future sessions as well.<br /> <br /> See the TSVN user's [http://tortoisesvn.tigris.org/list_etiquette.html mailing list information page] to subscribe to the mailing list. There are bunches of knowledgeable people there to help with virtually any problem you might have. Please make sure you follow proper nettiquette on this list, and research any questions you might have before posting something as a bug.<br /> <br /> ==RapidSVN==<br /> <br /> If you are using an operating system other than Windows, and you want a GUI client, there is a cross platform GUI client called [http://rapidsvn.tigris.org/ RapidSVN], but I have no experience, good or bad, with that program.<br /> <br /> == Subclipse ==<br /> <br /> There is also an [http://subclipse.tigris.org/ Eclipse plug-in] that is highly thought of, see [[Basic Developer Setup]] for details.</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Subversion_Setup_(deprecated)&diff=3243 2012-12-12T09:00:24Z

<p>Masaru20100: Added GNU/Linux details, reorganised the page a bit</p> <hr /> <div> {| align=&quot;right&quot;<br /> | __TOC__<br /> |}<br /> <br /> ==Introduction==<br /> <br /> Subversion is used as the version control system for the sources of the project — it holds all of the program source code and data that make up PCGen. It allows us to track changes in the various files, share our work with each other and to coordinate our activity in a single location.<br /> <br /> Developers use a subversion client to download the sources (called checkout) and update the sources with changes (called commit). The [http://subversion.apache.org/ official site] propose binaries or links to them for many operating systems. Most GNU/Linux distributions propose it as a package; in Debian it is the &lt;a href=&quot;apt:subversion&gt;subversion package&lt;/a&gt;. One commonly used Windows subversion client, [[#TortoiseSVN|TortoiseSVN]], is described below. Most integrated development environments (IDEs) include Subversion support. In the case of [[Eclipse]], it is by using the the [http://subclipse.tigris.org/servlets/ProjectProcess?pageID=p4wYuA subclipse] plugin.<br /> <br /> =NEW Update with SF Upgrade=<br /> <br /> On Beta projects (versus Classic projects), Sourceforge seems to be encouraging the use of svn+ssh rather than https. It seems that using http is still possible with Beta projects, maybe limited to read only access.<br /> <br /> The command to get the trunk and put it in the pcgen-trunk directory is:<br /> svn checkout https://svn.code.sf.net/p/pcgen/code/Trunk pcgen-trunk<br /> <br /> ==SVN+SSH Access==<br /> <br /> It is possible to use SSH to get the repository simply with read only access. The command to get the trunk is:<br /> svn checkout svn://svn.code.sf.net/p/pcgen/code/Trunk pcgen-trunk<br /> <br /> To get a read/write access, the command, where USERNAME has to be changed to your Sourceforge account id is:<br /> svn checkout --username=USERNAME svn+ssh://USERNAME@svn.code.sf.net/p/pcgen/code/Trunk pcgen-trunk<br /> <br /> It will be asking your Sourceforge password.<br /> <br /> It is also possible to use a SSH key to avoid sending your encrypted password over the network. It is more secure, because to access PCGen repository, they need a file on your computer, not just your password. If the SSH key file is protected by a password, the protection is more improved.<br /> The following paragraphs describe the procedure needed in order to use the key file.<br /> <br /> ===On Windows===<br /> <br /> '''Install PuttyGen'''<br /> <br /> puttygen.exe - http://the.earth.li/~sgtatham/putty/latest/x86/puttygen.exe<br /> <br /> <br /> '''Generate a SSH Key (file ends with .ppk)'''<br /> <br /> * run puttygen<br /> * Generate a New Key - SSH2<br /> * IN the comment section add 'sourceforgeUserId@shell.sourceforge.net'<br /> * OPTIONAL - You may set a passphrase, SF recommends it.<br /> * click the Save private key button<br /> * enter a filename for your new .ppk file when prompted<br /> * press Save<br /> * Copy the Public Key Information<br /> * Go to your SF Developer Account Services<br /> * Edit Keys<br /> * Paste your Public Key here<br /> <br /> <br /> '''Install Pageant'''<br /> <br /> pageant.exe - http://the.earth.li/~sgtatham/putty/latest/x86/pageant.exe<br /> <br /> <br /> '''Run the ageant'''<br /> <br /> * double-click on pageant.exe<br /> * you should now see a computer with a black hat in the tray<br /> * Add your .ppk to the ageant<br /> <br /> * right-click on the computer with the black hat in your taskbar<br /> * select Add Key from the context menu<br /> * browse to your recently-created .ppk file and select it<br /> * click Open<br /> * enter your password when prompted (If you set the Passphrase earlier in this set up for your key, this is what it is asking for)<br /> * verify that your key has been added by double-clicking on the computer with the black hat<br /> <br /> If you don't want to repeat this procedure after every reboot of your client, you should place Pageant in the auto-start group of your Windows installation. You can append the keys with complete paths as command line arguments to Pageant.exe.<br /> <br /> '''Install Plink'''<br /> <br /> plink.exe - http://the.earth.li/~sgtatham/putty/latest/x86/plink.exe<br /> <br /> <br /> '''Install TortoiseSVN'''<br /> <br /> TortoiseSVN download page<br /> <br /> <br /> '''Configure TortoiseSVN'''<br /> <br /> You need to tell TortoiseSVN to use TortoisePlink to handle SSH traffic. To do this see the following steps:<br /> <br /> * right-click on an explorer window somewhere<br /> * hover over TortoiseSVN in the context menu<br /> * select Settings from the sub-menu<br /> * select Network from the list on the left<br /> * under the SSH box in the right side, click the Browse... button<br /> * browse to and select TortoisePlink.exe (mine is in c:\Program Files\TortoiseSVN\bin\TortoisePlink.exe)<br /> * click Open<br /> * click Apply<br /> <br /> ===On GNU/Linux===<br /> <br /> On top of subversion client, an ssh client is needed. OpenSSH is included in most distributions, on Debian, the package is openssh-client.<br /> <br /> If you don’t have a key yet, create one with:<br /> ssh-keygen<br /> It will ask you where to store it (default is fine), and an eventual password. The public and private key will then be generated.<br /> <br /> This command is used to copy the public key to an SSH server (in the case of the default location to store the key):<br /> ssh-copy-id USERNAME@svn.code.sf.net<br /> <br /> Then you can use this command to grab the project:<br /> svn checkout --username=USERNAME svn+ssh://USERNAME@svn.code.sf.net/p/pcgen/code/Trunk pcgen-trunk<br /> <br /> In my case, my password to the id key was asked then my Sourceforge account key was asked.<br /> <br /> To be Done: testing commits with this configuration.<br /> <br /> ===On Mac OS X===<br /> <br /> [http://developer.apple.com/library/mac/#documentation/MacOSXServer/Conceptual/XServer_ProgrammingGuide/Articles/SSH.html Apple Developer SSH Page]<br /> <br /> <br /> =Related Articles=<br /> <br /> * Macintosh instructions can be found at [[SVN on the Mac]]<br /> * [[Web access to SVN]]<br /> * [[Merging]] with Subversion<br /> * [[Basic Developer Setup#Setting up with Eclipse|Setting up with Eclipse]]<br /> <br /> = Using SVN =<br /> <br /> The Subversion command line client is not terribly hard to use either, especially if you are familiar with CVS. You can download [http://subversion.tigris.org/project_packages.html binaries] for various platforms, including Windows.<br /> <br /> Note the section on that page that says that the Subversion development team does *not* directly support these binaries. They mean it. However, questions about these posted to the [http://subversion.tigris.org/servlets/ProjectMailingListList Subversion users mailing list] are usually answered pretty quickly.<br /> <br /> <br /> ==TortoiseSVN==<br /> <br /> TSVN is a GUI client for Subversion repositories. Before installing TSVN, please note that it is a Windows Explorer Shell extension, and as such, will not work on any other operating system.<br /> <br /> Make sure you have version 3.0 or later of the Windows Installer. This should be included in Windows Update for both Windows XP and Windows 2000. If for some reason you are still stuck on Windows 2000 (or earlier!), and can't update to the latest Service Pack, you are out of luck. I don't know if TSVN will work on Windows NT or Windows 98.<br /> <br /> # Close down all applications that might be running.<br /> # Download the [http://tortoisesvn.net/downloads TortoiseSVN Program] and the [http://tortoisesvn.sourceforge.net/?q=support manual].<br /> # Run the MSI.<br /> # Restart your computer.<br /> # Make sure you actually restarted your machine, not just logged off. TSVN is a shell extension, which means that you *must* restart your machine after installing.<br /> <br /> After you've installed TSVN, your Windows Explorer right-click context menus will all have a new entry called &quot;Tortoise SVN&quot;. Click this new entry to see the options available. Exactly how you will connect to a repository depends on how SourceForge has Subversion set up. I'll update this entry with more information when it becomes available.<br /> <br /> TSVN uses it's own SSH client, based on PuTTY, so you don't have to have it installed separately. The first time you attempt to browse a repository or do a check out, it will ask you for your credentials. You have the option of saving those credentials for future sessions as well.<br /> <br /> See the TSVN user's [http://tortoisesvn.tigris.org/list_etiquette.html mailing list information page] to subscribe to the mailing list. There are bunches of knowledgeable people there to help with virtually any problem you might have. Please make sure you follow proper nettiquette on this list, and research any questions you might have before posting something as a bug.<br /> <br /> ==RapidSVN==<br /> <br /> If you are using an operating system other than Windows, and you want a GUI client, there is a cross platform GUI client called [http://rapidsvn.tigris.org/ RapidSVN], but I have no experience, good or bad, with that program.<br /> <br /> == Subclipse ==<br /> <br /> There is also an [http://subclipse.tigris.org/ Eclipse plug-in] that is highly thought of, see [[Basic Developer Setup]] for details.</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Internationalization&diff=3151 2012-09-15T11:16:51Z

<p>Masaru20100: /* Application Packaging */</p> <hr /> <div>=Introduction=<br /> <br /> PCGen for 6.0+ releases will be attempting Internationalization (probably not in 6.0). Internalization is the process of changing software to allow it to be usable in any language and any locale. It is often abbreviated i18n. It is needed in PCGen because the software was not thought from the ground up to be usable from a language other than English. Localization is the process of creating the translations for a particular language, to do so the software need to able to handle internationalization. To deal with those, a Yahoo group called [http://groups.yahoo.com/group/pcgen_international pcgen_international] has been created.<br /> <br /> Requests for translations, problems with translations, and drafts of translated data sets or other translated content, as well as anything coming up with the Internationalization of PCGen are to be discussed in that group.<br /> <br /> Anybody interested to help out with translations to other languages, but also those who are just interested in the discussion of the matter, please join the group. Those interested in joining our efforts on the Internationalization of PCGen, please make yourself known on that list.<br /> <br /> This page is an attempt to regroup the information discussed on the list.<br /> <br /> There is a JIRA issue to regroup work on i18n: http://jira.pcgen.org/browse/CODE-733<br /> <br /> =Goal=<br /> <br /> The goal is to allow a user that doesn’t speak English to use the program in his native language with the specifics of his locale. This is an ideal goal, as it is not possible to provide all possible localizations with the low number of volunteers. What is needed is to provide the means for people to create localizations, in order to be able to do that, PCGen must be fully internationalized. Once this is done, the number of localization will slowly grow.<br /> <br /> PCGen interface, especially the part that are data set independent, are already internationalized and partially translated. Updating those translations is one of the needed step but not the only one.<br /> Another element is already internationalized, it’s the use of imperial or metric distance. There is also an option to change it.<br /> <br /> Other steps are need to attain internationalization.<br /> <br /> * First the '''installation process''' should be detailed in many languages, even if it’s only expanding a compressed file then using a script to launch the program.<br /> * All program '''UI strings''' should also be translatable, and so are the '''data strings'''. It seems that the non data strings are already translatable.<br /> * The skills should be '''sorted''' in the UI and in the output sheet according to the natural sorting of the language.<br /> * The '''output sheets''' should be in the user’s language too.<br /> * In case the user create it’s own translation, it should be possible to add the language in the option pane without having to compile the program (or worse have to change some code!). From a technical point of view, it also means that the translations should be external to the jar, or that a simple way to add a translation provided.<br /> <br /> * Saved characters are usually reusable with a new version of the program, and the il8n should not change that. (technical: the saved elements should stay the same, should probably be keys of some sort — I have no idea what the pcg files are)<br /> <br /> * It would be better for the '''measure and weights system''' to be initially set to a logical value rather than a set one, it would depend on the first chosen locale. It is also needed to verify that the imperial and metric system option are actually working.<br /> <br /> There is also other elements that would improve usability for the user, mainly that '''help/documentation''' would be available in his language. This is usually big work to be achieve but doesn’t need any change in PCGen.<br /> Another linked part is to be able to handle '''bug report''' in another language. To do that, the project need some people that would translate the bug reports to English before the development team is able to handle the bug reports. <br /> <br /> = Tools for internationalization and localization =<br /> <br /> To facilitate the internationalization and localization, some tools would be welcome. This tools are not necessary for the user, but are a great addition to maintainers.<br /> <br /> For internationalization:<br /> * a way to determine which strings are translatable or not. Some IDE and some code quality library provide this kind of tool.<br /> * a tool to eliminate (or point at) line that are not used any more in the main bundle file (it is quite big and there seems to bit not displayed text)<br /> <br /> For localization,<br /> * If a specific way of storing the data translation is used, a tool to produce and update a more standard file would be needed. In free project, gettext files (POT, PO) are often used. In Java, ResourceBundle are often used. Another standard format is XLIFF. Such a tool would be able to add new needed translation to a file of the standard format, and flag old translations. From the standard format, it would update the translation of the specific format file. This is needed because there is tool to ease and speed up translation, like Virtaal, ResourceBundle editors, or poedit, and they need a standard file format to be usable. <br /> * a tool test to see if some entries are not translated (maybe a page that would provide statistics) it might be more useful if UI and data separated<br /> * keeping glossary for each languages could help (some terms are found at many places)<br /> * A way to detect incorrect token translation (like using Dager instead of Dagger) is welcome. One way to do this is by generating error or warning when loading the translation file. It would be welcome to generate this list of error without having to launch PCGen, it would also be a nice addition to automatic testing.<br /> <br /> =Choices yet to be made=<br /> <br /> == Application Packaging ==<br /> <br /> As of December 2011, the application and translation of some part of the UI are bundled together.<br /> <br /> The question is should the application be bundled this way, or should there be language specific bundles.<br /> <br /> Pro of a all-in-one<br /> * user can change language on the fly<br /> * no need for explanations on how to install a new language set<br /> <br /> Cons<br /> * file could get quite big. But if it is text, it compress usually really well.<br /> <br /> == Genre class names ==<br /> <br /> In some languages, the class names can have a genre, like the feminine of priest is priestess in English (even if Cleric is the name of the class, it’s just to illustrate the point). Do we want to use those names in the class names or do we stick to whatever name the class is?<br /> <br /> = The goals in more details for PCGen =<br /> <br /> == Default language after clean install ==<br /> <br /> The language should be the system one. To the user, if doesn’t understand the default language, it will be a pain for him to find the option to change the language.<br /> It may be even better if the user is prompted with a choice at the start of the program (or at install) in order to simplify this process. It might be displayed only if the system language doesn’t exist in PCGen.<br /> As noted before, the measure and weight system used need to be chosen by the user at the first install with the default choice depending on the language/locale selected. After first launch or install, the option is changeable in the option preference panel.<br /> <br /> After clean install (no preference files):<br /> # PCGen determine the system language<br /> # if translation exist (and are believed good enough) switch to that language, if not, display an element to ask the user for the language to use.<br /> <br /> == Published language priority ==<br /> <br /> * The data that is not translated in a language, for example books not yet translated, should stay in the language it first get out.<br /> <br /> Books are almost never instantly translated by companies and some aren’t even translated. PCGen is also short on staff. All that means that the translations will probably be coming far after the original one.<br /> To an user that only use translated material, it will not be a problem.<br /> To an user that mix translated material, and not translated one, it shouldn’t become a problem.<br /> <br /> == Using different data languages ==<br /> <br /> * The data should be the same that the English one to avoid having to duplicate data correction.<br /> <br /> Having data in several languages could be a way to do il8n<br /> <br /> Cons<br /> * might need language specific bundles<br /> * a character used with one language cannot be seen by someone with a different language set<br /> * multiple corrections needed when correcting a bug (and if automatic generation of the translation is done, why not do it on the fly)<br /> <br /> It probably should be avoided to do it that way.<br /> <br /> == Separating data and UI translations ==<br /> <br /> * The interface language and the data language should be separate to allow people with book in a different language than their own to use the program more simply.<br /> To the user that speak only one language, it is no advantage nor a problem.<br /> For a user which use book that are not in the language he is most comfortable with, he would put the UI and the data to different languages. Same for someone who doesn’t use translation of the data but prefer to have a program in a language he is more comfortable with.<br /> <br /> == Custom content and localization ==<br /> <br /> * It should be possible to use another language in a data to allow non English speaker to develop custom content in another language. That seem to be a problem when combined with separate language from data. A way to avoid it is to provide easy reference to create custom content.<br /> <br /> This might be a hard to make real combined with other points.<br /> <br /> (might be wrong on this point [[User:Masaru20100|Masaru20100]]) At the moment, when creating custom content, the English token are used. For a non English-speaker, it is a problem, that’s why, ideally, he should be able to create custom by using his own language.<br /> <br /> The problem this cause is that if the user change the language of the program, the custom content either doesn’t work anymore, or all possible translations are checked which would cause massive slowness in the software.<br /> <br /> As it seams undoable, the user should be provided with way to easily find the needed English token for his custom content.<br /> <br /> == Indicating what is translated ==<br /> <br /> It would be good to provide the user with visual information of how much a dataset is translated in his language. It is not really a needed feature but could help him. If he knew that all is not translated, he would be less surprised.<br /> <br /> =Technical Aspects=<br /> <br /> == Language choice UI ==<br /> <br /> There is already options to change the language and the weights and measure system used.<br /> <br /> It might be best to have a drop down list rather than radio choices. New translation should appear in the list.<br /> <br /> == Formatting Issue ==<br /> <br /> Martijn pointed that number should be translatable. In Japanese, the classic number system is almost always used especially in RPG. In fact, translating numbers is not needed, what is needed is formatting numbers.<br /> <br /> That means that 10000 gets formatted 10,000 in English, 10 000 in French. In Japanese, it would either be 10,000 or 1万. There is even a locale to obtain 一万 if needed.<br /> <br /> In Java, this is usually done by using the NumberFormat class with the appropriate locale.<br /> <br /> == Sorting Issue ==<br /> <br /> Sorting is usually language dependent, meaning that the list of skills for example is not sorted the same way in different languages, even if the name are the same. The differences are usually minor between roman letter using languages.<br /> <br /> Actually, the international guys brought up the sorting thing again - another project would be to look into the GUI/core and replacing all the sorting of items with internationalized methods of sorting (Collators).<br /> <br /> Technically, Java already provide ways of sorting based on locale, it’s the Collator classes.<br /> <br /> == Unique Identifier ==<br /> <br /> Apparently there is already unique identifiers in use for items, so the idea would be to have a file with that id as the key and the translation as the value.<br /> <br /> == Output Sheet and Data Language ==<br /> <br /> The question is: should output sheets be locale dependent?<br /> If it is, when a sheet is corrected for one locale, it will need to be edited in all languages. Having languages taken into account in the same sheet make it more complicated.<br /> <br /> === Gender, race and level example ===<br /> <br /> There is also other problem as illustrated by the gender, race and level.<br /> <br /> In English, the gender propose Male, Female, Neuter, Unknown.<br /> In a English statblock, it is usually a line like: Gender Race Level N, as “Female drow cleric 3” for a drow noble [http://www.d20pfsrd.com/bestiary/monster-listings/humanoids/drow-common/drow-noble]. As of this writing, the software output “Female Drow Noble Cleric3”, or “Female Drow Noble Cleric 3” instead in most statblock sheets.<br /> The Unknown gender, and maybe the neuter one too, should probably not output as is in statblock sheets, while not <br /> On my system, with the preferences to use the system language, the female translation is used instead.<br /> <br /> In French, it is Mâle, Femelle, Neuter, Inconnu. The same noble drow is “Drow, prêtre 3” (prêtre is the translation of cleric) [http://www.pathfinder-fr.org/Wiki/Pathfinder-RPG.Drow%20noble.ashx]. I remember reading “Drow (f), prêtre 3”, where the sex is included by using a single letter in parenthesis. It could also have been “Drow, guerrière 3”, where guerrière is the feminine of fighter.<br /> As you can remark, the order is different.<br /> <br /> In Japanese, it looks like “ドラウ（女性）の貴族の3レベル・クレリック” [http://www29.atwiki.jp/prdj/pages/306.html#drow]. ドラウ is drow, 女性 means woman, の is the possessive, 貴族 is noble, レベル is level, クレリック is cleric. That would look like drow(female)’s noble’s 3rd level cleric. When the gender is unknown, it is not mentioned, goblin is such an example [http://www29.atwiki.jp/prdj/pages/244.html#goblin].<br /> <br /> In Italian, it seems to be “Drow nobile femmina Chierico 3”, ie Drow noble female Cleric 3. [http://prd.5clone.com/mostri/1982-drow]<br /> <br /> I think that in German, Female used as the choice and female as in female something is not written the same.<br /> <br /> Note that my examples uses Pathfinder because I do not know of translations of the SRD/RSRD online.<br /> <br /> This is only the first element of the stat block, and it seems already complicated. And there was no example of right-to-left languages, like Arabic or Hebrew. One thing seems common is that the gender value used on display, on the one used in stat blocks are different, whatever the language, and it is the same for race. For a standard character sheet, it seems that the UI value can be reused as is. In the case of gender, there is already two (three?) token, GENDER.SHORT and GENDER.LONG (and GENDER?). GENDER.SHORT, which has the same value as GENDER.LONG, could be changed to be the output value. A new value could be introduced, maybe something like GENDER.STATBLOCK.<br /> That still doesn’t handle the problem of the order of gender/race/class levels.<br /> <br /> === Data translation and output sheet values locale ===<br /> <br /> The issue is of what language to use when outputting the gender or other localized fields. At the moment, the data is in English, but there is a mix of English and whatever language the UI is defined to use. My primary example is Gender.<br /> Once a data language is introduced, it might make sense to use this language in the output sheets rather than the UI one (if they differ).<br /> <br /> == Tom’s technical proposal ==<br /> <br /> So looking back at the earlier thread, I never actually came back with my suggestion, so here is my perspective on localization and how it should be done. This has been made easier by some changes we have in the 6.x line, and having new UI (with what I believe to be a lot more isolation of code that does display of items) is a huge boost here to making this practical.<br /> <br /> First a few base facts as background:<br /> # For [almost] every item (Class, Skill, etc.) there is a unique identifier (generally referred to as a Key - note that if the &quot;KEY&quot; token is not used, then the Key is the name (first entry on the line in the data) (There is an exception to this we'll call [[#problem1|problem #1]])<br /> # There are basically 4 types of things that need translation:<br /> ## (2a) item names<br /> ## (2b) constants (like spell schools)<br /> ## (2c) variables (like meters/feet/etc.)<br /> ## (2d) Strings (like descriptions)<br /> ## [if anyone can think of more, let me know]<br /> # Most (but not all) tokens are &quot;unique&quot; or otherwise &quot;addressable&quot; in that they can only occur once per object. (Anyone notice that the tokens have started to be specific in the test code &amp; docs about whether they overwrite, add, etc. - this is one reason why - and yes, I've been slowly trying to make progress on this even in the 2007-2009 work) [The exception to this is [[#problem2|problem #2]])<br /> <br /> A few principles about l10n:<br /> # We must not make producing a data set materially more complicated than it is today (no requirement to put %L10NNAME% type gunk into data)<br /> # We should target an ability to tell us if l10n is complete for any given data set<br /> <br /> With that:<br /> * Following from #1, almost everything we have in the data today is &quot;addressable&quot;. By that, I mean that the OUTPUTNAME for a Skill called &quot;FooBar&quot; can be uniquely called out. The name hierarchy is something like: SKILL//Foo//OUTPUTNAME (exceptions are problems #[[#problem1|1]],[[#problem2|2]] to be addressed later)<br /> <br /> Given that, we can actually set principle #1 to &quot;The data remains unchanged&quot; (again, except for [[#problem2|problem #2]]). The entire data set is produced (assume US English for a moment). We then have a unique file for l10n that has things like:<br /> * SKILL:FooBar|Oobarf-ay<br /> * SKILL:FooBar:OUTPUTNAME|ooBarOutF-ay<br /> * etc.<br /> <br /> This (to the first order) covers 2a, 2d.<br /> <br /> For 2b items, we simply have to expand the list of items (SKILL, SPELL) that we are familiar with, so we get things like:<br /> * SPELLSCHOOL:Divination|Ivinationd-ay<br /> <br /> 2c is a bit more complicated, but I can't believe it's all that bad given it's a thing many applications already do (and it's a known thing)<br /> <br /> Each file could be named in such a way that it identifies it's l10n, e.g.:<br /> * srd_de_ch.l10n<br /> * ...and probably placed in a L10N subfolder of the initial dataset.<br /> * (Note I'd recommend we be clear on where these go, AND ALSO require that NO PCC FILES (I'd recommend we say no PCC or LST, but the formal limit would be no PCC) are recognized if they are in the L10N folder... so that the initial directory parse [which is probably one of the slower parts of our boot process] can immediately ignore the l10n directory and not have to look through the file list looking for .PCC files... alternately, we have a l10n folder that is parallel to the data folder, but that then adds complication in needing new preferences to point at multiple l10n folders and requires more complicated structure within that l10n structure to identify WHICH files in that folder map to which datasets... because we can't simply say a given English word will always translate the same way - English is way too overloaded for that.)<br /> <br /> I would recommend we keep things in a small subset of files, and not try to do 1:1 for each data file (that would produce a lot of file sprawl) - but that's not my call.<br /> <br /> Addressing principle #2:<br /> * Since we have a set of items we know would need to be covered (names, certain tokens), we should be able to load those into memory, and load the l10n file into memory and compare. This should be able to produce warnings of 2 kinds:<br /> * (W1) Errors where the base data set contains things that are not translated<br /> * (W2) Errors where the translation file attempts to translate things not in the base data set<br /> * I can't imagine that utility is all that hard to write (just would need to make the list of todos)<br /> <br /> This brings us to problems #1 and #2:<br /> <br /> &lt;a id=&quot;problem1&quot;&gt;Problem #1&lt;/a&gt;: Non-unique names<br /> * We glossed over this in 6.x, but the truth is that Spell names are not unique. Some of the *RDs have duplicate names (not all, but I forget which). Same is true for languages.<br /> * (1a) Spells can theoretically be differentiated by evaluating the &quot;TYPE&quot; token for Divine, Arcane, or Psionic (those are &quot;magical&quot; items in our code.<br /> * (1b) Languages can theoretically be differentiated between &quot;Spoken&quot; and &quot;Written&quot; as those are &quot;magical&quot; types.<br /> I believe both of those forms of magic are things on our backlog of FREQs to clean up... and the reason they are on the cleanup list really for L10N (as much as it is to just cleanup the overuse of TYPE)<br /> <br /> &lt;a id=&quot;problem2&quot;&gt;Problem #2&lt;/a&gt;: Non-unique tokens<br /> * There are only a few tokens that are not unique. DESC is one of them, if I recall correctly. The probably solution here is to simply give an identifier to each token. This might require a change to LST Something like:<br /> * DESC*Overall:x<br /> * DESC*Second:x<br /> * (Note: I'm not sure this syntax works or is by any means &quot;good&quot;. DESC:OVERALL|x might be better as it avoids potential issues with using * as a reserved character - take this is a principle of what would have to happen to DESC token, not as a full-blown proposal)<br /> <br /> Note that this naming of each DESC item (And other reusable items), while it breaks the &quot;can't change the data&quot; principle actually helps as much as it hurts... it would give us the ability to do things like:<br /> * DESC:.CLEARID.Overall<br /> * or things similar to that, which is actually a neat benefit for the small overhead of pain it puts into datasets. (Which, by the way, could be converted to whatever we decide on anyway with our nifty converter, so this doesn't seem all that bad)<br /> <br /> So in my mind, the question really is: Is going slightly more than half way good enough? There are areas where we could do translation, and some areas where we need some pretty material core code changes to support it.<br /> <br /> == Translation files ==<br /> <br /> Hi Tom,<br /> <br /> Welcome BACK!!!<br /> <br /> I don't mind the solutions put forth. Unique Identifiers for DESC is no more absurd then the Unique<br /> Identifiers we'll be needing to use more than one CHOOSER on a single line. Plus, the ability to<br /> clear off a Section of DESC would be awesome.<br /> <br /> I'd vote for as complete a job as possible.<br /> <br /> Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> thpr wrote:<br /> <br /> &gt; Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> Idea #1 (bad idea IMHO) is to put it in a separate directory:<br /> data/srd/...<br /> l10n/srd/...<br /> <br /> The problem with that is that you get into synchronization issues... so any time files/directory names change in one place they have to change in another. That's a contract on the data developer. (It would also require multiple l10n directories, since we support multiple data directories, so it's a bunch of code)<br /> <br /> Idea #2: Implicit Subdirectories:<br /> data/srd/*.lst<br /> data/srd/l10n/*.l10n<br /> <br /> That ensures that the l10n directory is associated with the dataset.<br /> <br /> Idea #3: Explicit designation:<br /> data/srd/srd.pcc contains LOCALIZATION:l10n/srd.l10n<br /> then:<br /> data/l10n/srd.l10n<br /> <br /> (would support more than one .l10n file)<br /> <br /> <br /> --- In pcgen_international@yahoogroups.com, Martijn Verburg &lt;martijnverburg@...&gt; wrote:<br /> &gt; &gt; (2) We should target an ability to tell us if l10n is complete for any<br /> &gt; &gt; given data set<br /> &gt; &gt;<br /> &gt; Not sure what you mean by this?<br /> <br /> If there is an object called &quot;Dagger&quot; we need to ensure the file contains:<br /> EQUIPMENT:Dagger|Aggerd-ay<br /> <br /> If it contains:<br /> EQUIPMENT:Dgager|Aggerd-ay<br /> <br /> That is also an error (just like the &quot;unconstructed reference&quot; items are errors. By capturing both type 1 and type 2 error (things that aren't translated as well as things that shouldn't have been translated) we are capturing the vast majority of the simple problems.<br /> <br /> === Also need translation ===<br /> <br /> Page number: either the original book is referenced, but it doesn’t help people that have the translation where the page is not the same, or the translated book is referenced and that number need to be localized.<br /> Usually data sets represents individual books, but there is sometimes compilation done of those. That means that several dataset would represent a single one. I know the French editor of Pathfinder does this. No idea what the content is and how it is organized, but it might work by changing the description to include something like chapter X of book Z in the description.<br /> <br /> The data sets points to the English books. Should the translation points to the localized one, if it exists?<br /> <br /> == See also ==<br /> <br /> * [[Localization]]</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Internationalization&diff=3150 2012-09-15T11:15:24Z

<p>Masaru20100: </p> <hr /> <div>=Introduction=<br /> <br /> PCGen for 6.0+ releases will be attempting Internationalization (probably not in 6.0). Internalization is the process of changing software to allow it to be usable in any language and any locale. It is often abbreviated i18n. It is needed in PCGen because the software was not thought from the ground up to be usable from a language other than English. Localization is the process of creating the translations for a particular language, to do so the software need to able to handle internationalization. To deal with those, a Yahoo group called [http://groups.yahoo.com/group/pcgen_international pcgen_international] has been created.<br /> <br /> Requests for translations, problems with translations, and drafts of translated data sets or other translated content, as well as anything coming up with the Internationalization of PCGen are to be discussed in that group.<br /> <br /> Anybody interested to help out with translations to other languages, but also those who are just interested in the discussion of the matter, please join the group. Those interested in joining our efforts on the Internationalization of PCGen, please make yourself known on that list.<br /> <br /> This page is an attempt to regroup the information discussed on the list.<br /> <br /> There is a JIRA issue to regroup work on i18n: http://jira.pcgen.org/browse/CODE-733<br /> <br /> =Goal=<br /> <br /> The goal is to allow a user that doesn’t speak English to use the program in his native language with the specifics of his locale. This is an ideal goal, as it is not possible to provide all possible localizations with the low number of volunteers. What is needed is to provide the means for people to create localizations, in order to be able to do that, PCGen must be fully internationalized. Once this is done, the number of localization will slowly grow.<br /> <br /> PCGen interface, especially the part that are data set independent, are already internationalized and partially translated. Updating those translations is one of the needed step but not the only one.<br /> Another element is already internationalized, it’s the use of imperial or metric distance. There is also an option to change it.<br /> <br /> Other steps are need to attain internationalization.<br /> <br /> * First the '''installation process''' should be detailed in many languages, even if it’s only expanding a compressed file then using a script to launch the program.<br /> * All program '''UI strings''' should also be translatable, and so are the '''data strings'''. It seems that the non data strings are already translatable.<br /> * The skills should be '''sorted''' in the UI and in the output sheet according to the natural sorting of the language.<br /> * The '''output sheets''' should be in the user’s language too.<br /> * In case the user create it’s own translation, it should be possible to add the language in the option pane without having to compile the program (or worse have to change some code!). From a technical point of view, it also means that the translations should be external to the jar, or that a simple way to add a translation provided.<br /> <br /> * Saved characters are usually reusable with a new version of the program, and the il8n should not change that. (technical: the saved elements should stay the same, should probably be keys of some sort — I have no idea what the pcg files are)<br /> <br /> * It would be better for the '''measure and weights system''' to be initially set to a logical value rather than a set one, it would depend on the first chosen locale. It is also needed to verify that the imperial and metric system option are actually working.<br /> <br /> There is also other elements that would improve usability for the user, mainly that '''help/documentation''' would be available in his language. This is usually big work to be achieve but doesn’t need any change in PCGen.<br /> Another linked part is to be able to handle '''bug report''' in another language. To do that, the project need some people that would translate the bug reports to English before the development team is able to handle the bug reports. <br /> <br /> = Tools for internationalization and localization =<br /> <br /> To facilitate the internationalization and localization, some tools would be welcome. This tools are not necessary for the user, but are a great addition to maintainers.<br /> <br /> For internationalization:<br /> * a way to determine which strings are translatable or not. Some IDE and some code quality library provide this kind of tool.<br /> * a tool to eliminate (or point at) line that are not used any more in the main bundle file (it is quite big and there seems to bit not displayed text)<br /> <br /> For localization,<br /> * If a specific way of storing the data translation is used, a tool to produce and update a more standard file would be needed. In free project, gettext files (POT, PO) are often used. In Java, ResourceBundle are often used. Another standard format is XLIFF. Such a tool would be able to add new needed translation to a file of the standard format, and flag old translations. From the standard format, it would update the translation of the specific format file. This is needed because there is tool to ease and speed up translation, like Virtaal, ResourceBundle editors, or poedit, and they need a standard file format to be usable. <br /> * a tool test to see if some entries are not translated (maybe a page that would provide statistics) it might be more useful if UI and data separated<br /> * keeping glossary for each languages could help (some terms are found at many places)<br /> * A way to detect incorrect token translation (like using Dager instead of Dagger) is welcome. One way to do this is by generating error or warning when loading the translation file. It would be welcome to generate this list of error without having to launch PCGen, it would also be a nice addition to automatic testing.<br /> <br /> =Choices yet to be made=<br /> <br /> == Application Packaging ==<br /> <br /> As of December 2011, the application and translation of some part of the UI are bundled together.<br /> <br /> The question is should the application be bundled this way, or should there be language specific bundles.<br /> <br /> Pro of a all-in-one<br /> * user can change language on the fly<br /> * no need for explanations on how to install a new language set<br /> <br /> Cons<br /> * file could get quite big<br /> <br /> == Genre class names ==<br /> <br /> In some languages, the class names can have a genre, like the feminine of priest is priestess in English (even if Cleric is the name of the class, it’s just to illustrate the point). Do we want to use those names in the class names or do we stick to whatever name the class is?<br /> <br /> = The goals in more details for PCGen =<br /> <br /> == Default language after clean install ==<br /> <br /> The language should be the system one. To the user, if doesn’t understand the default language, it will be a pain for him to find the option to change the language.<br /> It may be even better if the user is prompted with a choice at the start of the program (or at install) in order to simplify this process. It might be displayed only if the system language doesn’t exist in PCGen.<br /> As noted before, the measure and weight system used need to be chosen by the user at the first install with the default choice depending on the language/locale selected. After first launch or install, the option is changeable in the option preference panel.<br /> <br /> After clean install (no preference files):<br /> # PCGen determine the system language<br /> # if translation exist (and are believed good enough) switch to that language, if not, display an element to ask the user for the language to use.<br /> <br /> == Published language priority ==<br /> <br /> * The data that is not translated in a language, for example books not yet translated, should stay in the language it first get out.<br /> <br /> Books are almost never instantly translated by companies and some aren’t even translated. PCGen is also short on staff. All that means that the translations will probably be coming far after the original one.<br /> To an user that only use translated material, it will not be a problem.<br /> To an user that mix translated material, and not translated one, it shouldn’t become a problem.<br /> <br /> == Using different data languages ==<br /> <br /> * The data should be the same that the English one to avoid having to duplicate data correction.<br /> <br /> Having data in several languages could be a way to do il8n<br /> <br /> Cons<br /> * might need language specific bundles<br /> * a character used with one language cannot be seen by someone with a different language set<br /> * multiple corrections needed when correcting a bug (and if automatic generation of the translation is done, why not do it on the fly)<br /> <br /> It probably should be avoided to do it that way.<br /> <br /> == Separating data and UI translations ==<br /> <br /> * The interface language and the data language should be separate to allow people with book in a different language than their own to use the program more simply.<br /> To the user that speak only one language, it is no advantage nor a problem.<br /> For a user which use book that are not in the language he is most comfortable with, he would put the UI and the data to different languages. Same for someone who doesn’t use translation of the data but prefer to have a program in a language he is more comfortable with.<br /> <br /> == Custom content and localization ==<br /> <br /> * It should be possible to use another language in a data to allow non English speaker to develop custom content in another language. That seem to be a problem when combined with separate language from data. A way to avoid it is to provide easy reference to create custom content.<br /> <br /> This might be a hard to make real combined with other points.<br /> <br /> (might be wrong on this point [[User:Masaru20100|Masaru20100]]) At the moment, when creating custom content, the English token are used. For a non English-speaker, it is a problem, that’s why, ideally, he should be able to create custom by using his own language.<br /> <br /> The problem this cause is that if the user change the language of the program, the custom content either doesn’t work anymore, or all possible translations are checked which would cause massive slowness in the software.<br /> <br /> As it seams undoable, the user should be provided with way to easily find the needed English token for his custom content.<br /> <br /> == Indicating what is translated ==<br /> <br /> It would be good to provide the user with visual information of how much a dataset is translated in his language. It is not really a needed feature but could help him. If he knew that all is not translated, he would be less surprised.<br /> <br /> =Technical Aspects=<br /> <br /> == Language choice UI ==<br /> <br /> There is already options to change the language and the weights and measure system used.<br /> <br /> It might be best to have a drop down list rather than radio choices. New translation should appear in the list.<br /> <br /> == Formatting Issue ==<br /> <br /> Martijn pointed that number should be translatable. In Japanese, the classic number system is almost always used especially in RPG. In fact, translating numbers is not needed, what is needed is formatting numbers.<br /> <br /> That means that 10000 gets formatted 10,000 in English, 10 000 in French. In Japanese, it would either be 10,000 or 1万. There is even a locale to obtain 一万 if needed.<br /> <br /> In Java, this is usually done by using the NumberFormat class with the appropriate locale.<br /> <br /> == Sorting Issue ==<br /> <br /> Sorting is usually language dependent, meaning that the list of skills for example is not sorted the same way in different languages, even if the name are the same. The differences are usually minor between roman letter using languages.<br /> <br /> Actually, the international guys brought up the sorting thing again - another project would be to look into the GUI/core and replacing all the sorting of items with internationalized methods of sorting (Collators).<br /> <br /> Technically, Java already provide ways of sorting based on locale, it’s the Collator classes.<br /> <br /> == Unique Identifier ==<br /> <br /> Apparently there is already unique identifiers in use for items, so the idea would be to have a file with that id as the key and the translation as the value.<br /> <br /> == Output Sheet and Data Language ==<br /> <br /> The question is: should output sheets be locale dependent?<br /> If it is, when a sheet is corrected for one locale, it will need to be edited in all languages. Having languages taken into account in the same sheet make it more complicated.<br /> <br /> === Gender, race and level example ===<br /> <br /> There is also other problem as illustrated by the gender, race and level.<br /> <br /> In English, the gender propose Male, Female, Neuter, Unknown.<br /> In a English statblock, it is usually a line like: Gender Race Level N, as “Female drow cleric 3” for a drow noble [http://www.d20pfsrd.com/bestiary/monster-listings/humanoids/drow-common/drow-noble]. As of this writing, the software output “Female Drow Noble Cleric3”, or “Female Drow Noble Cleric 3” instead in most statblock sheets.<br /> The Unknown gender, and maybe the neuter one too, should probably not output as is in statblock sheets, while not <br /> On my system, with the preferences to use the system language, the female translation is used instead.<br /> <br /> In French, it is Mâle, Femelle, Neuter, Inconnu. The same noble drow is “Drow, prêtre 3” (prêtre is the translation of cleric) [http://www.pathfinder-fr.org/Wiki/Pathfinder-RPG.Drow%20noble.ashx]. I remember reading “Drow (f), prêtre 3”, where the sex is included by using a single letter in parenthesis. It could also have been “Drow, guerrière 3”, where guerrière is the feminine of fighter.<br /> As you can remark, the order is different.<br /> <br /> In Japanese, it looks like “ドラウ（女性）の貴族の3レベル・クレリック” [http://www29.atwiki.jp/prdj/pages/306.html#drow]. ドラウ is drow, 女性 means woman, の is the possessive, 貴族 is noble, レベル is level, クレリック is cleric. That would look like drow(female)’s noble’s 3rd level cleric. When the gender is unknown, it is not mentioned, goblin is such an example [http://www29.atwiki.jp/prdj/pages/244.html#goblin].<br /> <br /> In Italian, it seems to be “Drow nobile femmina Chierico 3”, ie Drow noble female Cleric 3. [http://prd.5clone.com/mostri/1982-drow]<br /> <br /> I think that in German, Female used as the choice and female as in female something is not written the same.<br /> <br /> Note that my examples uses Pathfinder because I do not know of translations of the SRD/RSRD online.<br /> <br /> This is only the first element of the stat block, and it seems already complicated. And there was no example of right-to-left languages, like Arabic or Hebrew. One thing seems common is that the gender value used on display, on the one used in stat blocks are different, whatever the language, and it is the same for race. For a standard character sheet, it seems that the UI value can be reused as is. In the case of gender, there is already two (three?) token, GENDER.SHORT and GENDER.LONG (and GENDER?). GENDER.SHORT, which has the same value as GENDER.LONG, could be changed to be the output value. A new value could be introduced, maybe something like GENDER.STATBLOCK.<br /> That still doesn’t handle the problem of the order of gender/race/class levels.<br /> <br /> === Data translation and output sheet values locale ===<br /> <br /> The issue is of what language to use when outputting the gender or other localized fields. At the moment, the data is in English, but there is a mix of English and whatever language the UI is defined to use. My primary example is Gender.<br /> Once a data language is introduced, it might make sense to use this language in the output sheets rather than the UI one (if they differ).<br /> <br /> == Tom’s technical proposal ==<br /> <br /> So looking back at the earlier thread, I never actually came back with my suggestion, so here is my perspective on localization and how it should be done. This has been made easier by some changes we have in the 6.x line, and having new UI (with what I believe to be a lot more isolation of code that does display of items) is a huge boost here to making this practical.<br /> <br /> First a few base facts as background:<br /> # For [almost] every item (Class, Skill, etc.) there is a unique identifier (generally referred to as a Key - note that if the &quot;KEY&quot; token is not used, then the Key is the name (first entry on the line in the data) (There is an exception to this we'll call [[#problem1|problem #1]])<br /> # There are basically 4 types of things that need translation:<br /> ## (2a) item names<br /> ## (2b) constants (like spell schools)<br /> ## (2c) variables (like meters/feet/etc.)<br /> ## (2d) Strings (like descriptions)<br /> ## [if anyone can think of more, let me know]<br /> # Most (but not all) tokens are &quot;unique&quot; or otherwise &quot;addressable&quot; in that they can only occur once per object. (Anyone notice that the tokens have started to be specific in the test code &amp; docs about whether they overwrite, add, etc. - this is one reason why - and yes, I've been slowly trying to make progress on this even in the 2007-2009 work) [The exception to this is [[#problem2|problem #2]])<br /> <br /> A few principles about l10n:<br /> # We must not make producing a data set materially more complicated than it is today (no requirement to put %L10NNAME% type gunk into data)<br /> # We should target an ability to tell us if l10n is complete for any given data set<br /> <br /> With that:<br /> * Following from #1, almost everything we have in the data today is &quot;addressable&quot;. By that, I mean that the OUTPUTNAME for a Skill called &quot;FooBar&quot; can be uniquely called out. The name hierarchy is something like: SKILL//Foo//OUTPUTNAME (exceptions are problems #[[#problem1|1]],[[#problem2|2]] to be addressed later)<br /> <br /> Given that, we can actually set principle #1 to &quot;The data remains unchanged&quot; (again, except for [[#problem2|problem #2]]). The entire data set is produced (assume US English for a moment). We then have a unique file for l10n that has things like:<br /> * SKILL:FooBar|Oobarf-ay<br /> * SKILL:FooBar:OUTPUTNAME|ooBarOutF-ay<br /> * etc.<br /> <br /> This (to the first order) covers 2a, 2d.<br /> <br /> For 2b items, we simply have to expand the list of items (SKILL, SPELL) that we are familiar with, so we get things like:<br /> * SPELLSCHOOL:Divination|Ivinationd-ay<br /> <br /> 2c is a bit more complicated, but I can't believe it's all that bad given it's a thing many applications already do (and it's a known thing)<br /> <br /> Each file could be named in such a way that it identifies it's l10n, e.g.:<br /> * srd_de_ch.l10n<br /> * ...and probably placed in a L10N subfolder of the initial dataset.<br /> * (Note I'd recommend we be clear on where these go, AND ALSO require that NO PCC FILES (I'd recommend we say no PCC or LST, but the formal limit would be no PCC) are recognized if they are in the L10N folder... so that the initial directory parse [which is probably one of the slower parts of our boot process] can immediately ignore the l10n directory and not have to look through the file list looking for .PCC files... alternately, we have a l10n folder that is parallel to the data folder, but that then adds complication in needing new preferences to point at multiple l10n folders and requires more complicated structure within that l10n structure to identify WHICH files in that folder map to which datasets... because we can't simply say a given English word will always translate the same way - English is way too overloaded for that.)<br /> <br /> I would recommend we keep things in a small subset of files, and not try to do 1:1 for each data file (that would produce a lot of file sprawl) - but that's not my call.<br /> <br /> Addressing principle #2:<br /> * Since we have a set of items we know would need to be covered (names, certain tokens), we should be able to load those into memory, and load the l10n file into memory and compare. This should be able to produce warnings of 2 kinds:<br /> * (W1) Errors where the base data set contains things that are not translated<br /> * (W2) Errors where the translation file attempts to translate things not in the base data set<br /> * I can't imagine that utility is all that hard to write (just would need to make the list of todos)<br /> <br /> This brings us to problems #1 and #2:<br /> <br /> &lt;a id=&quot;problem1&quot;&gt;Problem #1&lt;/a&gt;: Non-unique names<br /> * We glossed over this in 6.x, but the truth is that Spell names are not unique. Some of the *RDs have duplicate names (not all, but I forget which). Same is true for languages.<br /> * (1a) Spells can theoretically be differentiated by evaluating the &quot;TYPE&quot; token for Divine, Arcane, or Psionic (those are &quot;magical&quot; items in our code.<br /> * (1b) Languages can theoretically be differentiated between &quot;Spoken&quot; and &quot;Written&quot; as those are &quot;magical&quot; types.<br /> I believe both of those forms of magic are things on our backlog of FREQs to clean up... and the reason they are on the cleanup list really for L10N (as much as it is to just cleanup the overuse of TYPE)<br /> <br /> &lt;a id=&quot;problem2&quot;&gt;Problem #2&lt;/a&gt;: Non-unique tokens<br /> * There are only a few tokens that are not unique. DESC is one of them, if I recall correctly. The probably solution here is to simply give an identifier to each token. This might require a change to LST Something like:<br /> * DESC*Overall:x<br /> * DESC*Second:x<br /> * (Note: I'm not sure this syntax works or is by any means &quot;good&quot;. DESC:OVERALL|x might be better as it avoids potential issues with using * as a reserved character - take this is a principle of what would have to happen to DESC token, not as a full-blown proposal)<br /> <br /> Note that this naming of each DESC item (And other reusable items), while it breaks the &quot;can't change the data&quot; principle actually helps as much as it hurts... it would give us the ability to do things like:<br /> * DESC:.CLEARID.Overall<br /> * or things similar to that, which is actually a neat benefit for the small overhead of pain it puts into datasets. (Which, by the way, could be converted to whatever we decide on anyway with our nifty converter, so this doesn't seem all that bad)<br /> <br /> So in my mind, the question really is: Is going slightly more than half way good enough? There are areas where we could do translation, and some areas where we need some pretty material core code changes to support it.<br /> <br /> == Translation files ==<br /> <br /> Hi Tom,<br /> <br /> Welcome BACK!!!<br /> <br /> I don't mind the solutions put forth. Unique Identifiers for DESC is no more absurd then the Unique<br /> Identifiers we'll be needing to use more than one CHOOSER on a single line. Plus, the ability to<br /> clear off a Section of DESC would be awesome.<br /> <br /> I'd vote for as complete a job as possible.<br /> <br /> Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> thpr wrote:<br /> <br /> &gt; Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> Idea #1 (bad idea IMHO) is to put it in a separate directory:<br /> data/srd/...<br /> l10n/srd/...<br /> <br /> The problem with that is that you get into synchronization issues... so any time files/directory names change in one place they have to change in another. That's a contract on the data developer. (It would also require multiple l10n directories, since we support multiple data directories, so it's a bunch of code)<br /> <br /> Idea #2: Implicit Subdirectories:<br /> data/srd/*.lst<br /> data/srd/l10n/*.l10n<br /> <br /> That ensures that the l10n directory is associated with the dataset.<br /> <br /> Idea #3: Explicit designation:<br /> data/srd/srd.pcc contains LOCALIZATION:l10n/srd.l10n<br /> then:<br /> data/l10n/srd.l10n<br /> <br /> (would support more than one .l10n file)<br /> <br /> <br /> --- In pcgen_international@yahoogroups.com, Martijn Verburg &lt;martijnverburg@...&gt; wrote:<br /> &gt; &gt; (2) We should target an ability to tell us if l10n is complete for any<br /> &gt; &gt; given data set<br /> &gt; &gt;<br /> &gt; Not sure what you mean by this?<br /> <br /> If there is an object called &quot;Dagger&quot; we need to ensure the file contains:<br /> EQUIPMENT:Dagger|Aggerd-ay<br /> <br /> If it contains:<br /> EQUIPMENT:Dgager|Aggerd-ay<br /> <br /> That is also an error (just like the &quot;unconstructed reference&quot; items are errors. By capturing both type 1 and type 2 error (things that aren't translated as well as things that shouldn't have been translated) we are capturing the vast majority of the simple problems.<br /> <br /> === Also need translation ===<br /> <br /> Page number: either the original book is referenced, but it doesn’t help people that have the translation where the page is not the same, or the translated book is referenced and that number need to be localized.<br /> Usually data sets represents individual books, but there is sometimes compilation done of those. That means that several dataset would represent a single one. I know the French editor of Pathfinder does this. No idea what the content is and how it is organized, but it might work by changing the description to include something like chapter X of book Z in the description.<br /> <br /> The data sets points to the English books. Should the translation points to the localized one, if it exists?<br /> <br /> == See also ==<br /> <br /> * [[Localization]]</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Internationalization&diff=3149 2012-09-15T11:00:02Z

<p>Masaru20100: /* Tom’s technical proposal */ added link/anchors</p> <hr /> <div>=Introduction=<br /> <br /> PCGen for 6.0+ releases will be attempting Internationalization (probably not in 6.0). Internalization is the process of changing software to allow it to be usable in any language and any locale. It is often abbreviated i18n. It is needed in PCGen because the software was not thought from the ground up to be usable from a language other than English. Localization is the process of creating the translations for a particular language, to do so the software need to able to handle internationalization. To deal with those, a Yahoo group called [http://groups.yahoo.com/group/pcgen_international pcgen_international] has been created.<br /> <br /> Requests for translations, problems with translations, and drafts of translated data sets or other translated content, as well as anything coming up with the Internationalization of PCGen are to be discussed in that group.<br /> <br /> Anybody interested to help out with translations to other languages, but also those who are just interested in the discussion of the matter, please join the group. Those interested in joining our efforts on the Internationalization of PCGen, please make yourself known on that list.<br /> <br /> This page is an attempt to regroup the information discussed on the list.<br /> <br /> There is a JIRA issue to regroup work on i18n: http://jira.pcgen.org/browse/CODE-733<br /> <br /> =Goal=<br /> <br /> The goal is to allow a user that doesn’t speak English to use the program in his native language with the specifics of his locale. This is an ideal goal, as it is not possible to provide all possible localizations with the low number of volunteers. What is needed is to provide the means for people to create localizations, in order to be able to do that, PCGen must be fully internationalized. Once this is done, the number of localization will slowly grow.<br /> <br /> PCGen interface, especially the part that are data set independent, are already internationalized and partially translated. Updating those translations is one of the needed step but not the only one.<br /> Another element is already internationalized, it’s the use of imperial or metric distance. There is also an option to change it.<br /> <br /> Other steps are need to attain internationalization.<br /> <br /> * First the '''installation process''' should be detailed in many languages, even if it’s only expanding a compressed file then using a script to launch the program.<br /> * All program '''UI strings''' should also be translatable, and so are the '''data strings'''. It seems that the non data strings are already translatable.<br /> * The skills should be '''sorted''' in the UI and in the output sheet according to the natural sorting of the language.<br /> * The '''output sheets''' should be in the user’s language too.<br /> * In case the user create it’s own translation, it should be possible to add the language in the option pane without having to compile the program (or worse have to change some code!). From a technical point of view, it also means that the translations should be external to the jar, or that a simple way to add a translation provided.<br /> <br /> * Saved characters are usually reusable with a new version of the program, and the il8n should not change that. (technical: the saved elements should stay the same, should probably be keys of some sort — I have no idea what the pcg files are)<br /> <br /> * It would be better for the '''measure and weights system''' to be initially set to a logical value rather than a set one, it would depend on the first chosen locale. It is also needed to verify that the imperial and metric system option are actually working.<br /> <br /> There is also other elements that would improve usability for the user, mainly that '''help/documentation''' would be available in his language. This is usually big work to be achieve but doesn’t need any change in PCGen.<br /> Another linked part is to be able to handle '''bug report''' in another language. To do that, the project need some people that would translate the bug reports to English before the development team is able to handle the bug reports. <br /> <br /> = Tools for internationalization and localization =<br /> <br /> To facilitate the internationalization and localization, some tools would be welcome. This tools are not necessary for the user, but are a great addition to maintainers.<br /> <br /> For internationalization:<br /> * a way to determine which strings are translatable or not. Some IDE and some code quality library provide this kind of tool.<br /> * a tool to eliminate (or point at) line that are not used any more in the main bundle file (it is quite big and there seems to bit not displayed text)<br /> <br /> For localization,<br /> * If a specific way of storing the data translation is used, a tool to produce and update a more standard file would be needed. In free project, gettext files (POT, PO) are often used. In Java, ResourceBundle are often used. Another standard format is XLIFF. Such a tool would be able to add new needed translation to a file of the standard format, and flag old translations. From the standard format, it would update the translation of the specific format file. This is needed because there is tool to ease and speed up translation, like Virtaal, ResourceBundle editors, or poedit, and they need a standard file format to be usable. <br /> * a tool test to see if some entries are not translated (maybe a page that would provide statistics) it might be more useful if UI and data separated<br /> * keeping glossary for each languages could help (some terms are found at many places)<br /> * A way to detect incorrect token translation (like using Dager instead of Dagger) is welcome. One way to do this is by generating error or warning when loading the translation file. It would be welcome to generate this list of error without having to launch PCGen, it would also be a nice addition to automatic testing.<br /> <br /> =Choices yet to be made=<br /> <br /> == Application Packaging ==<br /> <br /> As of December 2011, the application and translation of some part of the UI are bundled together.<br /> <br /> The question is should the application be bundled this way, or should there be language specific bundles.<br /> <br /> Pro of a all-in-one<br /> * user can change language on the fly<br /> * no need for explanations on how to install a new language set<br /> <br /> Cons<br /> * file could get quite big<br /> <br /> == Genre class names ==<br /> <br /> In some languages, the class names can have a genre, like the feminine of priest is priestess in English (even if Cleric is the name of the class, it’s just to illustrate the point). Do we want to use those names in the class names or do we stick to whatever name the class is?<br /> <br /> = The goals in more details for PCGen =<br /> <br /> == Default language after clean install ==<br /> <br /> The language should be the system one. To the user, if doesn’t understand the default language, it will be a pain for him to find the option to change the language.<br /> It may be even better if the user is prompted with a choice at the start of the program (or at install) in order to simplify this process. It might be displayed only if the system language doesn’t exist in PCGen.<br /> As noted before, the measure and weight system used need to be chosen by the user at the first install with the default choice depending on the language/locale selected. After first launch or install, the option is changeable in the option preference panel.<br /> <br /> After clean install (no preference files):<br /> # PCGen determine the system language<br /> # if translation exist (and are believed good enough) switch to that language, if not, display an element to ask the user for the language to use.<br /> <br /> == Published language priority ==<br /> <br /> * The data that is not translated in a language, for example books not yet translated, should stay in the language it first get out.<br /> <br /> Books are almost never instantly translated by companies and some aren’t even translated. PCGen is also short on staff. All that means that the translations will probably be coming far after the original one.<br /> To an user that only use translated material, it will not be a problem.<br /> To an user that mix translated material, and not translated one, it shouldn’t become a problem.<br /> <br /> == Using different data languages ==<br /> <br /> * The data should be the same that the English one to avoid having to duplicate data correction.<br /> <br /> Having data in several languages could be a way to do il8n<br /> <br /> Cons<br /> * might need language specific bundles<br /> * a character used with one language cannot be seen by someone with a different language set<br /> * multiple corrections needed when correcting a bug (and if automatic generation of the translation is done, why not do it on the fly)<br /> <br /> It probably should be avoided to do it that way.<br /> <br /> == Separating data and UI translations ==<br /> <br /> * The interface language and the data language should be separate to allow people with book in a different language than their own to use the program more simply.<br /> To the user that speak only one language, it is no advantage nor a problem.<br /> For a user which use book that are not in the language he is most comfortable with, he would put the UI and the data to different languages. Same for someone who doesn’t use translation of the data but prefer to have a program in a language he is more comfortable with.<br /> <br /> == Custom content and localization ==<br /> <br /> * It should be possible to use another language in a data to allow non English speaker to develop custom content in another language. That seem to be a problem when combined with separate language from data. A way to avoid it is to provide easy reference to create custom content.<br /> <br /> This might be a hard to make real combined with other points.<br /> <br /> (might be wrong on this point [[User:Masaru20100|Masaru20100]]) At the moment, when creating custom content, the English token are used. For a non English-speaker, it is a problem, that’s why, ideally, he should be able to create custom by using his own language.<br /> <br /> The problem this cause is that if the user change the language of the program, the custom content either doesn’t work anymore, or all possible translations are checked which would cause massive slowness in the software.<br /> <br /> As it seams undoable, the user should be provided with way to easily find the needed English token for his custom content.<br /> <br /> == Indicating what is translated ==<br /> <br /> It would be good to provide the user with visual information of how much a dataset is translated in his language. It is not really a needed feature but could help him. If he knew that all is not translated, he would be less surprised.<br /> <br /> =Technical Aspects=<br /> <br /> == Language choice UI ==<br /> <br /> There is already options to change the language and the weights and measure system used.<br /> <br /> It might be best to have a drop down list rather than radio choices. New translation should appear in the list.<br /> <br /> == Formatting Issue ==<br /> <br /> Martijn pointed that number should be translatable. In Japanese, the classic number system is almost always used especially in RPG. In fact, translating numbers is not needed, what is needed is formatting numbers.<br /> <br /> That means that 10000 gets formatted 10,000 in English, 10 000 in French. In Japanese, it would either be 10,000 or 1万. There is even a locale to obtain 一万 if needed.<br /> <br /> In Java, this is usually done by using the NumberFormat class with the appropriate locale.<br /> <br /> == Sorting Issue ==<br /> <br /> Sorting is usually language dependent, meaning that the list of skills for example is not sorted the same way in different languages, even if the name are the same. The differences are usually minor between roman letter using languages.<br /> <br /> Actually, the international guys brought up the sorting thing again - another project would be to look into the GUI/core and replacing all the sorting of items with internationalized methods of sorting (Collators).<br /> <br /> Technically, Java already provide ways of sorting based on locale, it’s the Collator classes.<br /> <br /> == Unique Identifier ==<br /> <br /> Apparently there is already unique identifiers in use for items, so the idea would be to have a file with that id as the key and the translation as the value.<br /> <br /> == Output Sheet and Data Language ==<br /> <br /> The question is: should output sheets be locale dependent?<br /> If it is, when a sheet is corrected for one locale, it will need to be edited in all languages. Having languages taken into account in the same sheet make it more complicated.<br /> <br /> === Gender, race and level example ===<br /> <br /> There is also other problem as illustrated by the gender, race and level.<br /> <br /> In English, the gender propose Male, Female, Neuter, Unknown.<br /> In a English statblock, it is usually a line like: Gender Race Level N, as “Female drow cleric 3” for a drow noble [http://www.d20pfsrd.com/bestiary/monster-listings/humanoids/drow-common/drow-noble]. As of this writing, the software output “Female Drow Noble Cleric3”, or “Female Drow Noble Cleric 3” instead in most statblock sheets.<br /> The Unknown gender, and maybe the neuter one too, should probably not output as is in statblock sheets, while not <br /> On my system, with the preferences to use the system language, the female translation is used instead.<br /> <br /> In French, it is Mâle, Femelle, Neuter, Inconnu. The same noble drow is “Drow, prêtre 3” (prêtre is the translation of cleric) [http://www.pathfinder-fr.org/Wiki/Pathfinder-RPG.Drow%20noble.ashx]. I remember reading “Drow (f), prêtre 3”, where the sex is included by using a single letter in parenthesis. It could also have been “Drow, guerrière 3”, where guerrière is the feminine of fighter.<br /> As you can remark, the order is different.<br /> <br /> In Japanese, it looks like “ドラウ（女性）の貴族の3レベル・クレリック” [http://www29.atwiki.jp/prdj/pages/306.html#drow]. ドラウ is drow, 女性 means woman, の is the possessive, 貴族 is noble, レベル is level, クレリック is cleric. That would look like drow(female)’s noble’s 3rd level cleric. When the gender is unknown, it is not mentioned, goblin is such an example [http://www29.atwiki.jp/prdj/pages/244.html#goblin].<br /> <br /> In Italian, it seems to be “Drow nobile femmina Chierico 3”, ie Drow noble female Cleric 3. [http://prd.5clone.com/mostri/1982-drow]<br /> <br /> I think that in German, Female used as the choice and female as in female something is not written the same.<br /> <br /> Note that my examples uses Pathfinder because I do not know of translations of the SRD/RSRD online.<br /> <br /> This is only the first element of the stat block, and it seems already complicated. And there was no example of right-to-left languages, like Arabic or Hebrew. One thing seems common is that the gender value used on display, on the one used in stat blocks are different, whatever the language, and it is the same for race. For a standard character sheet, it seems that the UI value can be reused as is. In the case of gender, there is already two (three?) token, GENDER.SHORT and GENDER.LONG (and GENDER?). GENDER.SHORT, which has the same value as GENDER.LONG, could be changed to be the output value. A new value could be introduced, maybe something like GENDER.STATBLOCK.<br /> That still doesn’t handle the problem of the order of gender/race/class levels.<br /> <br /> === Data translation and output sheet values locale ===<br /> <br /> The issue is of what language to use when outputting the gender or other localized fields. At the moment, the data is in English, but there is a mix of English and whatever language the UI is defined to use. My primary example is Gender.<br /> Once a data language is introduced, it might make sense to use this language in the output sheets rather than the UI one (if they differ).<br /> <br /> == Tom’s technical proposal ==<br /> <br /> So looking back at the earlier thread, I never actually came back with my suggestion, so here is my perspective on localization and how it should be done. This has been made easier by some changes we have in the 6.x line, and having new UI (with what I believe to be a lot more isolation of code that does display of items) is a huge boost here to making this practical.<br /> <br /> First a few base facts as background:<br /> # For [almost] every item (Class, Skill, etc.) there is a unique identifier (generally referred to as a Key - note that if the &quot;KEY&quot; token is not used, then the Key is the name (first entry on the line in the data) (There is an exception to this we'll call [[#problem1|problem #1]])<br /> # There are basically 4 types of things that need translation:<br /> ## (2a) item names<br /> ## (2b) constants (like spell schools)<br /> ## (2c) variables (like meters/feet/etc.)<br /> ## (2d) Strings (like descriptions)<br /> ## [if anyone can think of more, let me know]<br /> # Most (but not all) tokens are &quot;unique&quot; or otherwise &quot;addressable&quot; in that they can only occur once per object. (Anyone notice that the tokens have started to be specific in the test code &amp; docs about whether they overwrite, add, etc. - this is one reason why - and yes, I've been slowly trying to make progress on this even in the 2007-2009 work) [The exception to this is [[#problem2|problem #2]])<br /> <br /> A few principles about l10n:<br /> # We must not make producing a data set materially more complicated than it is today (no requirement to put %L10NNAME% type gunk into data)<br /> # We should target an ability to tell us if l10n is complete for any given data set<br /> <br /> With that:<br /> * Following from #1, almost everything we have in the data today is &quot;addressable&quot;. By that, I mean that the OUTPUTNAME for a Skill called &quot;FooBar&quot; can be uniquely called out. The name hierarchy is something like: SKILL//Foo//OUTPUTNAME (exceptions are problems #[[#problem1|1]],[[#problem2|2]] to be addressed later)<br /> <br /> Given that, we can actually set principle #1 to &quot;The data remains unchanged&quot; (again, except for [[#problem2|problem #2]]). The entire data set is produced (assume US English for a moment). We then have a unique file for l10n that has things like:<br /> * SKILL:FooBar|Oobarf-ay<br /> * SKILL:FooBar:OUTPUTNAME|ooBarOutF-ay<br /> * etc.<br /> <br /> This (to the first order) covers 2a, 2d.<br /> <br /> For 2b items, we simply have to expand the list of items (SKILL, SPELL) that we are familiar with, so we get things like:<br /> * SPELLSCHOOL:Divination|Ivinationd-ay<br /> <br /> 2c is a bit more complicated, but I can't believe it's all that bad given it's a thing many applications already do (and it's a known thing)<br /> <br /> Each file could be named in such a way that it identifies it's l10n, e.g.:<br /> * srd_de_ch.l10n<br /> * ...and probably placed in a L10N subfolder of the initial dataset.<br /> * (Note I'd recommend we be clear on where these go, AND ALSO require that NO PCC FILES (I'd recommend we say no PCC or LST, but the formal limit would be no PCC) are recognized if they are in the L10N folder... so that the initial directory parse [which is probably one of the slower parts of our boot process] can immediately ignore the l10n directory and not have to look through the file list looking for .PCC files... alternately, we have a l10n folder that is parallel to the data folder, but that then adds complication in needing new preferences to point at multiple l10n folders and requires more complicated structure within that l10n structure to identify WHICH files in that folder map to which datasets... because we can't simply say a given English word will always translate the same way - English is way too overloaded for that.)<br /> <br /> I would recommend we keep things in a small subset of files, and not try to do 1:1 for each data file (that would produce a lot of file sprawl) - but that's not my call.<br /> <br /> Addressing principle #2:<br /> * Since we have a set of items we know would need to be covered (names, certain tokens), we should be able to load those into memory, and load the l10n file into memory and compare. This should be able to produce warnings of 2 kinds:<br /> * (W1) Errors where the base data set contains things that are not translated<br /> * (W2) Errors where the translation file attempts to translate things not in the base data set<br /> * I can't imagine that utility is all that hard to write (just would need to make the list of todos)<br /> <br /> This brings us to problems #1 and #2:<br /> <br /> &lt;a id=&quot;problem1&quot;&gt;Problem #1&lt;/a&gt;: Non-unique names<br /> * We glossed over this in 6.x, but the truth is that Spell names are not unique. Some of the *RDs have duplicate names (not all, but I forget which). Same is true for languages.<br /> * (1a) Spells can theoretically be differentiated by evaluating the &quot;TYPE&quot; token for Divine, Arcane, or Psionic (those are &quot;magical&quot; items in our code.<br /> * (1b) Languages can theoretically be differentiated between &quot;Spoken&quot; and &quot;Written&quot; as those are &quot;magical&quot; types.<br /> I believe both of those forms of magic are things on our backlog of FREQs to clean up... and the reason they are on the cleanup list really for L10N (as much as it is to just cleanup the overuse of TYPE)<br /> <br /> &lt;a id=&quot;problem2&quot;&gt;Problem #2&lt;/a&gt;: Non-unique tokens<br /> * There are only a few tokens that are not unique. DESC is one of them, if I recall correctly. The probably solution here is to simply give an identifier to each token. This might require a change to LST Something like:<br /> * DESC*Overall:x<br /> * DESC*Second:x<br /> * (Note: I'm not sure this syntax works or is by any means &quot;good&quot;. DESC:OVERALL|x might be better as it avoids potential issues with using * as a reserved character - take this is a principle of what would have to happen to DESC token, not as a full-blown proposal)<br /> <br /> Note that this naming of each DESC item (And other reusable items), while it breaks the &quot;can't change the data&quot; principle actually helps as much as it hurts... it would give us the ability to do things like:<br /> * DESC:.CLEARID.Overall<br /> * or things similar to that, which is actually a neat benefit for the small overhead of pain it puts into datasets. (Which, by the way, could be converted to whatever we decide on anyway with our nifty converter, so this doesn't seem all that bad)<br /> <br /> So in my mind, the question really is: Is going slightly more than half way good enough? There are areas where we could do translation, and some areas where we need some pretty material core code changes to support it.<br /> <br /> == Translation files ==<br /> <br /> Hi Tom,<br /> <br /> Welcome BACK!!!<br /> <br /> I don't mind the solutions put forth. Unique Identifiers for DESC is no more absurd then the Unique<br /> Identifiers we'll be needing to use more than one CHOOSER on a single line. Plus, the ability to<br /> clear off a Section of DESC would be awesome.<br /> <br /> I'd vote for as complete a job as possible.<br /> <br /> Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> thpr wrote:<br /> <br /> &gt; Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> Idea #1 (bad idea IMHO) is to put it in a separate directory:<br /> data/srd/...<br /> l10n/srd/...<br /> <br /> The problem with that is that you get into synchronization issues... so any time files/directory names change in one place they have to change in another. That's a contract on the data developer. (It would also require multiple l10n directories, since we support multiple data directories, so it's a bunch of code)<br /> <br /> Idea #2: Implicit Subdirectories:<br /> data/srd/*.lst<br /> data/srd/l10n/*.l10n<br /> <br /> That ensures that the l10n directory is associated with the dataset.<br /> <br /> Idea #3: Explicit designation:<br /> data/srd/srd.pcc contains LOCALIZATION:l10n/srd.l10n<br /> then:<br /> data/l10n/srd.l10n<br /> <br /> (would support more than one .l10n file)<br /> <br /> <br /> --- In pcgen_international@yahoogroups.com, Martijn Verburg &lt;martijnverburg@...&gt; wrote:<br /> &gt; &gt; (2) We should target an ability to tell us if l10n is complete for any<br /> &gt; &gt; given data set<br /> &gt; &gt;<br /> &gt; Not sure what you mean by this?<br /> <br /> If there is an object called &quot;Dagger&quot; we need to ensure the file contains:<br /> EQUIPMENT:Dagger|Aggerd-ay<br /> <br /> If it contains:<br /> EQUIPMENT:Dgager|Aggerd-ay<br /> <br /> That is also an error (just like the &quot;unconstructed reference&quot; items are errors. By capturing both type 1 and type 2 error (things that aren't translated as well as things that shouldn't have been translated) we are capturing the vast majority of the simple problems.<br /> <br /> == See also ==<br /> <br /> * [[Localization]]</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Internationalization&diff=3148 2012-09-15T10:45:44Z

<p>Masaru20100: /* Output Sheet and Data Language */</p> <hr /> <div>=Introduction=<br /> <br /> PCGen for 6.0+ releases will be attempting Internationalization (probably not in 6.0). Internalization is the process of changing software to allow it to be usable in any language and any locale. It is often abbreviated i18n. It is needed in PCGen because the software was not thought from the ground up to be usable from a language other than English. Localization is the process of creating the translations for a particular language, to do so the software need to able to handle internationalization. To deal with those, a Yahoo group called [http://groups.yahoo.com/group/pcgen_international pcgen_international] has been created.<br /> <br /> Requests for translations, problems with translations, and drafts of translated data sets or other translated content, as well as anything coming up with the Internationalization of PCGen are to be discussed in that group.<br /> <br /> Anybody interested to help out with translations to other languages, but also those who are just interested in the discussion of the matter, please join the group. Those interested in joining our efforts on the Internationalization of PCGen, please make yourself known on that list.<br /> <br /> This page is an attempt to regroup the information discussed on the list.<br /> <br /> There is a JIRA issue to regroup work on i18n: http://jira.pcgen.org/browse/CODE-733<br /> <br /> =Goal=<br /> <br /> The goal is to allow a user that doesn’t speak English to use the program in his native language with the specifics of his locale. This is an ideal goal, as it is not possible to provide all possible localizations with the low number of volunteers. What is needed is to provide the means for people to create localizations, in order to be able to do that, PCGen must be fully internationalized. Once this is done, the number of localization will slowly grow.<br /> <br /> PCGen interface, especially the part that are data set independent, are already internationalized and partially translated. Updating those translations is one of the needed step but not the only one.<br /> Another element is already internationalized, it’s the use of imperial or metric distance. There is also an option to change it.<br /> <br /> Other steps are need to attain internationalization.<br /> <br /> * First the '''installation process''' should be detailed in many languages, even if it’s only expanding a compressed file then using a script to launch the program.<br /> * All program '''UI strings''' should also be translatable, and so are the '''data strings'''. It seems that the non data strings are already translatable.<br /> * The skills should be '''sorted''' in the UI and in the output sheet according to the natural sorting of the language.<br /> * The '''output sheets''' should be in the user’s language too.<br /> * In case the user create it’s own translation, it should be possible to add the language in the option pane without having to compile the program (or worse have to change some code!). From a technical point of view, it also means that the translations should be external to the jar, or that a simple way to add a translation provided.<br /> <br /> * Saved characters are usually reusable with a new version of the program, and the il8n should not change that. (technical: the saved elements should stay the same, should probably be keys of some sort — I have no idea what the pcg files are)<br /> <br /> * It would be better for the '''measure and weights system''' to be initially set to a logical value rather than a set one, it would depend on the first chosen locale. It is also needed to verify that the imperial and metric system option are actually working.<br /> <br /> There is also other elements that would improve usability for the user, mainly that '''help/documentation''' would be available in his language. This is usually big work to be achieve but doesn’t need any change in PCGen.<br /> Another linked part is to be able to handle '''bug report''' in another language. To do that, the project need some people that would translate the bug reports to English before the development team is able to handle the bug reports. <br /> <br /> = Tools for internationalization and localization =<br /> <br /> To facilitate the internationalization and localization, some tools would be welcome. This tools are not necessary for the user, but are a great addition to maintainers.<br /> <br /> For internationalization:<br /> * a way to determine which strings are translatable or not. Some IDE and some code quality library provide this kind of tool.<br /> * a tool to eliminate (or point at) line that are not used any more in the main bundle file (it is quite big and there seems to bit not displayed text)<br /> <br /> For localization,<br /> * If a specific way of storing the data translation is used, a tool to produce and update a more standard file would be needed. In free project, gettext files (POT, PO) are often used. In Java, ResourceBundle are often used. Another standard format is XLIFF. Such a tool would be able to add new needed translation to a file of the standard format, and flag old translations. From the standard format, it would update the translation of the specific format file. This is needed because there is tool to ease and speed up translation, like Virtaal, ResourceBundle editors, or poedit, and they need a standard file format to be usable. <br /> * a tool test to see if some entries are not translated (maybe a page that would provide statistics) it might be more useful if UI and data separated<br /> * keeping glossary for each languages could help (some terms are found at many places)<br /> * A way to detect incorrect token translation (like using Dager instead of Dagger) is welcome. One way to do this is by generating error or warning when loading the translation file. It would be welcome to generate this list of error without having to launch PCGen, it would also be a nice addition to automatic testing.<br /> <br /> =Choices yet to be made=<br /> <br /> == Application Packaging ==<br /> <br /> As of December 2011, the application and translation of some part of the UI are bundled together.<br /> <br /> The question is should the application be bundled this way, or should there be language specific bundles.<br /> <br /> Pro of a all-in-one<br /> * user can change language on the fly<br /> * no need for explanations on how to install a new language set<br /> <br /> Cons<br /> * file could get quite big<br /> <br /> == Genre class names ==<br /> <br /> In some languages, the class names can have a genre, like the feminine of priest is priestess in English (even if Cleric is the name of the class, it’s just to illustrate the point). Do we want to use those names in the class names or do we stick to whatever name the class is?<br /> <br /> = The goals in more details for PCGen =<br /> <br /> == Default language after clean install ==<br /> <br /> The language should be the system one. To the user, if doesn’t understand the default language, it will be a pain for him to find the option to change the language.<br /> It may be even better if the user is prompted with a choice at the start of the program (or at install) in order to simplify this process. It might be displayed only if the system language doesn’t exist in PCGen.<br /> As noted before, the measure and weight system used need to be chosen by the user at the first install with the default choice depending on the language/locale selected. After first launch or install, the option is changeable in the option preference panel.<br /> <br /> After clean install (no preference files):<br /> # PCGen determine the system language<br /> # if translation exist (and are believed good enough) switch to that language, if not, display an element to ask the user for the language to use.<br /> <br /> == Published language priority ==<br /> <br /> * The data that is not translated in a language, for example books not yet translated, should stay in the language it first get out.<br /> <br /> Books are almost never instantly translated by companies and some aren’t even translated. PCGen is also short on staff. All that means that the translations will probably be coming far after the original one.<br /> To an user that only use translated material, it will not be a problem.<br /> To an user that mix translated material, and not translated one, it shouldn’t become a problem.<br /> <br /> == Using different data languages ==<br /> <br /> * The data should be the same that the English one to avoid having to duplicate data correction.<br /> <br /> Having data in several languages could be a way to do il8n<br /> <br /> Cons<br /> * might need language specific bundles<br /> * a character used with one language cannot be seen by someone with a different language set<br /> * multiple corrections needed when correcting a bug (and if automatic generation of the translation is done, why not do it on the fly)<br /> <br /> It probably should be avoided to do it that way.<br /> <br /> == Separating data and UI translations ==<br /> <br /> * The interface language and the data language should be separate to allow people with book in a different language than their own to use the program more simply.<br /> To the user that speak only one language, it is no advantage nor a problem.<br /> For a user which use book that are not in the language he is most comfortable with, he would put the UI and the data to different languages. Same for someone who doesn’t use translation of the data but prefer to have a program in a language he is more comfortable with.<br /> <br /> == Custom content and localization ==<br /> <br /> * It should be possible to use another language in a data to allow non English speaker to develop custom content in another language. That seem to be a problem when combined with separate language from data. A way to avoid it is to provide easy reference to create custom content.<br /> <br /> This might be a hard to make real combined with other points.<br /> <br /> (might be wrong on this point [[User:Masaru20100|Masaru20100]]) At the moment, when creating custom content, the English token are used. For a non English-speaker, it is a problem, that’s why, ideally, he should be able to create custom by using his own language.<br /> <br /> The problem this cause is that if the user change the language of the program, the custom content either doesn’t work anymore, or all possible translations are checked which would cause massive slowness in the software.<br /> <br /> As it seams undoable, the user should be provided with way to easily find the needed English token for his custom content.<br /> <br /> == Indicating what is translated ==<br /> <br /> It would be good to provide the user with visual information of how much a dataset is translated in his language. It is not really a needed feature but could help him. If he knew that all is not translated, he would be less surprised.<br /> <br /> =Technical Aspects=<br /> <br /> == Language choice UI ==<br /> <br /> There is already options to change the language and the weights and measure system used.<br /> <br /> It might be best to have a drop down list rather than radio choices. New translation should appear in the list.<br /> <br /> == Formatting Issue ==<br /> <br /> Martijn pointed that number should be translatable. In Japanese, the classic number system is almost always used especially in RPG. In fact, translating numbers is not needed, what is needed is formatting numbers.<br /> <br /> That means that 10000 gets formatted 10,000 in English, 10 000 in French. In Japanese, it would either be 10,000 or 1万. There is even a locale to obtain 一万 if needed.<br /> <br /> In Java, this is usually done by using the NumberFormat class with the appropriate locale.<br /> <br /> == Sorting Issue ==<br /> <br /> Sorting is usually language dependent, meaning that the list of skills for example is not sorted the same way in different languages, even if the name are the same. The differences are usually minor between roman letter using languages.<br /> <br /> Actually, the international guys brought up the sorting thing again - another project would be to look into the GUI/core and replacing all the sorting of items with internationalized methods of sorting (Collators).<br /> <br /> Technically, Java already provide ways of sorting based on locale, it’s the Collator classes.<br /> <br /> == Unique Identifier ==<br /> <br /> Apparently there is already unique identifiers in use for items, so the idea would be to have a file with that id as the key and the translation as the value.<br /> <br /> == Output Sheet and Data Language ==<br /> <br /> The question is: should output sheets be locale dependent?<br /> If it is, when a sheet is corrected for one locale, it will need to be edited in all languages. Having languages taken into account in the same sheet make it more complicated.<br /> <br /> === Gender, race and level example ===<br /> <br /> There is also other problem as illustrated by the gender, race and level.<br /> <br /> In English, the gender propose Male, Female, Neuter, Unknown.<br /> In a English statblock, it is usually a line like: Gender Race Level N, as “Female drow cleric 3” for a drow noble [http://www.d20pfsrd.com/bestiary/monster-listings/humanoids/drow-common/drow-noble]. As of this writing, the software output “Female Drow Noble Cleric3”, or “Female Drow Noble Cleric 3” instead in most statblock sheets.<br /> The Unknown gender, and maybe the neuter one too, should probably not output as is in statblock sheets, while not <br /> On my system, with the preferences to use the system language, the female translation is used instead.<br /> <br /> In French, it is Mâle, Femelle, Neuter, Inconnu. The same noble drow is “Drow, prêtre 3” (prêtre is the translation of cleric) [http://www.pathfinder-fr.org/Wiki/Pathfinder-RPG.Drow%20noble.ashx]. I remember reading “Drow (f), prêtre 3”, where the sex is included by using a single letter in parenthesis. It could also have been “Drow, guerrière 3”, where guerrière is the feminine of fighter.<br /> As you can remark, the order is different.<br /> <br /> In Japanese, it looks like “ドラウ（女性）の貴族の3レベル・クレリック” [http://www29.atwiki.jp/prdj/pages/306.html#drow]. ドラウ is drow, 女性 means woman, の is the possessive, 貴族 is noble, レベル is level, クレリック is cleric. That would look like drow(female)’s noble’s 3rd level cleric. When the gender is unknown, it is not mentioned, goblin is such an example [http://www29.atwiki.jp/prdj/pages/244.html#goblin].<br /> <br /> In Italian, it seems to be “Drow nobile femmina Chierico 3”, ie Drow noble female Cleric 3. [http://prd.5clone.com/mostri/1982-drow]<br /> <br /> I think that in German, Female used as the choice and female as in female something is not written the same.<br /> <br /> Note that my examples uses Pathfinder because I do not know of translations of the SRD/RSRD online.<br /> <br /> This is only the first element of the stat block, and it seems already complicated. And there was no example of right-to-left languages, like Arabic or Hebrew. One thing seems common is that the gender value used on display, on the one used in stat blocks are different, whatever the language, and it is the same for race. For a standard character sheet, it seems that the UI value can be reused as is. In the case of gender, there is already two (three?) token, GENDER.SHORT and GENDER.LONG (and GENDER?). GENDER.SHORT, which has the same value as GENDER.LONG, could be changed to be the output value. A new value could be introduced, maybe something like GENDER.STATBLOCK.<br /> That still doesn’t handle the problem of the order of gender/race/class levels.<br /> <br /> === Data translation and output sheet values locale ===<br /> <br /> The issue is of what language to use when outputting the gender or other localized fields. At the moment, the data is in English, but there is a mix of English and whatever language the UI is defined to use. My primary example is Gender.<br /> Once a data language is introduced, it might make sense to use this language in the output sheets rather than the UI one (if they differ).<br /> <br /> == Tom’s technical proposal ==<br /> <br /> So looking back at the earlier thread, I never actually came back with my suggestion, so here is my perspective on localization and how it should be done. This has been made easier by some changes we have in the 6.x line, and having new UI (with what I believe to be a lot more isolation of code that does display of items) is a huge boost here to making this practical.<br /> <br /> First a few base facts as background:<br /> # For [almost] every item (Class, Skill, etc.) there is a unique identifier (generally referred to as a Key - note that if the &quot;KEY&quot; token is not used, then the Key is the name (first entry on the line in the data) (There is an exception to this we'll call problem #1)<br /> # There are basically 4 types of things that need translation:<br /> ## (2a) item names<br /> ## (2b) constants (like spell schools)<br /> ## (2c) variables (like meters/feet/etc.)<br /> ## (2d) Strings (like descriptions)<br /> ## [if anyone can think of more, let me know]<br /> # Most (but not all) tokens are &quot;unique&quot; or otherwise &quot;addressable&quot; in that they can only occur once per object. (Anyone notice that the tokens have started to be specific in the test code &amp; docs about whether they overwrite, add, etc. - this is one reason why - and yes, I've been slowly trying to make progress on this even in the 2007-2009 work) [The exception to this is problem #2)<br /> <br /> A few principles about l10n:<br /> # We must not make producing a data set materially more complicated than it is today (no requirement to put %L10NNAME% type gunk into data)<br /> # We should target an ability to tell us if l10n is complete for any given data set<br /> <br /> With that:<br /> * Following from #1, almost everything we have in the data today is &quot;addressable&quot;. By that, I mean that the OUTPUTNAME for a Skill called &quot;FooBar&quot; can be uniquely called out. The name hierarchy is something like: SKILL//Foo//OUTPUTNAME (exceptions are problems #1,2 to be addressed later)<br /> <br /> Given that, we can actually set principle #1 to &quot;The data remains unchanged&quot; (again, except for problem #2). The entire data set is produced (assume US English for a moment). We then have a unique file for l10n that has things like:<br /> * SKILL:FooBar|Oobarf-ay<br /> * SKILL:FooBar:OUTPUTNAME|ooBarOutF-ay<br /> * etc.<br /> <br /> This (to the first order) covers 2a, 2d.<br /> <br /> For 2b items, we simply have to expand the list of items (SKILL, SPELL) that we are familiar with, so we get things like:<br /> * SPELLSCHOOL:Divination|Ivinationd-ay<br /> <br /> 2c is a bit more complicated, but I can't believe it's all that bad given it's a thing many applications already do (and it's a known thing)<br /> <br /> Each file could be named in such a way that it identifies it's l10n, e.g.:<br /> * srd_de_ch.l10n<br /> * ...and probably placed in a L10N subfolder of the initial dataset.<br /> * (Note I'd recommend we be clear on where these go, AND ALSO require that NO PCC FILES (I'd recommend we say no PCC or LST, but the formal limit would be no PCC) are recognized if they are in the L10N folder... so that the initial directory parse [which is probably one of the slower parts of our boot process] can immediately ignore the l10n directory and not have to look through the file list looking for .PCC files... alternately, we have a l10n folder that is parallel to the data folder, but that then adds complication in needing new preferences to point at multiple l10n folders and requires more complicated structure within that l10n structure to identify WHICH files in that folder map to which datasets... because we can't simply say a given English word will always translate the same way - English is way too overloaded for that.)<br /> <br /> I would recommend we keep things in a small subset of files, and not try to do 1:1 for each data file (that would produce a lot of file sprawl) - but that's not my call.<br /> <br /> Addressing principle #2:<br /> * Since we have a set of items we know would need to be covered (names, certain tokens), we should be able to load those into memory, and load the l10n file into memory and compare. This should be able to produce warnings of 2 kinds:<br /> * (W1) Errors where the base data set contains things that are not translated<br /> * (W2) Errors where the translation file attempts to translate things not in the base data set<br /> * I can't imagine that utility is all that hard to write (just would need to make the list of todos)<br /> <br /> This brings us to problems #1 and #2:<br /> <br /> Problem #1: Non-unique names<br /> * We glossed over this in 6.x, but the truth is that Spell names are not unique. Some of the *RDs have duplicate names (not all, but I forget which). Same is true for languages.<br /> * (1a) Spells can theoretically be differentiated by evaluating the &quot;TYPE&quot; token for Divine, Arcane, or Psionic (those are &quot;magical&quot; items in our code.<br /> * (1b) Languages can theoretically be differentiated between &quot;Spoken&quot; and &quot;Written&quot; as those are &quot;magical&quot; types.<br /> I believe both of those forms of magic are things on our backlog of FREQs to clean up... and the reason they are on the cleanup list really for L10N (as much as it is to just cleanup the overuse of TYPE)<br /> <br /> Problem #2: Non-unique tokens<br /> * There are only a few tokens that are not unique. DESC is one of them, if I recall correctly. The probably solution here is to simply give an identifier to each token. This might require a change to LST Something like:<br /> * DESC*Overall:x<br /> * DESC*Second:x<br /> * (Note: I'm not sure this syntax works or is by any means &quot;good&quot;. DESC:OVERALL|x might be better as it avoids potential issues with using * as a reserved character - take this is a principle of what would have to happen to DESC token, not as a full-blown proposal)<br /> <br /> Note that this naming of each DESC item (And other reusable items), while it breaks the &quot;can't change the data&quot; principle actually helps as much as it hurts... it would give us the ability to do things like:<br /> * DESC:.CLEARID.Overall<br /> * or things similar to that, which is actually a neat benefit for the small overhead of pain it puts into datasets. (Which, by the way, could be converted to whatever we decide on anyway with our nifty converter, so this doesn't seem all that bad)<br /> <br /> So in my mind, the question really is: Is going slightly more than half way good enough? There are areas where we could do translation, and some areas where we need some pretty material core code changes to support it.<br /> <br /> == Translation files ==<br /> <br /> Hi Tom,<br /> <br /> Welcome BACK!!!<br /> <br /> I don't mind the solutions put forth. Unique Identifiers for DESC is no more absurd then the Unique<br /> Identifiers we'll be needing to use more than one CHOOSER on a single line. Plus, the ability to<br /> clear off a Section of DESC would be awesome.<br /> <br /> I'd vote for as complete a job as possible.<br /> <br /> Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> thpr wrote:<br /> <br /> &gt; Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> Idea #1 (bad idea IMHO) is to put it in a separate directory:<br /> data/srd/...<br /> l10n/srd/...<br /> <br /> The problem with that is that you get into synchronization issues... so any time files/directory names change in one place they have to change in another. That's a contract on the data developer. (It would also require multiple l10n directories, since we support multiple data directories, so it's a bunch of code)<br /> <br /> Idea #2: Implicit Subdirectories:<br /> data/srd/*.lst<br /> data/srd/l10n/*.l10n<br /> <br /> That ensures that the l10n directory is associated with the dataset.<br /> <br /> Idea #3: Explicit designation:<br /> data/srd/srd.pcc contains LOCALIZATION:l10n/srd.l10n<br /> then:<br /> data/l10n/srd.l10n<br /> <br /> (would support more than one .l10n file)<br /> <br /> <br /> --- In pcgen_international@yahoogroups.com, Martijn Verburg &lt;martijnverburg@...&gt; wrote:<br /> &gt; &gt; (2) We should target an ability to tell us if l10n is complete for any<br /> &gt; &gt; given data set<br /> &gt; &gt;<br /> &gt; Not sure what you mean by this?<br /> <br /> If there is an object called &quot;Dagger&quot; we need to ensure the file contains:<br /> EQUIPMENT:Dagger|Aggerd-ay<br /> <br /> If it contains:<br /> EQUIPMENT:Dgager|Aggerd-ay<br /> <br /> That is also an error (just like the &quot;unconstructed reference&quot; items are errors. By capturing both type 1 and type 2 error (things that aren't translated as well as things that shouldn't have been translated) we are capturing the vast majority of the simple problems.<br /> <br /> == See also ==<br /> <br /> * [[Localization]]</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Internationalization&diff=3147 2012-09-15T10:23:27Z

<p>Masaru20100: /* Technical Aspects */ added more technical problems</p> <hr /> <div>=Introduction=<br /> <br /> PCGen for 6.0+ releases will be attempting Internationalization (probably not in 6.0). Internalization is the process of changing software to allow it to be usable in any language and any locale. It is often abbreviated i18n. It is needed in PCGen because the software was not thought from the ground up to be usable from a language other than English. Localization is the process of creating the translations for a particular language, to do so the software need to able to handle internationalization. To deal with those, a Yahoo group called [http://groups.yahoo.com/group/pcgen_international pcgen_international] has been created.<br /> <br /> Requests for translations, problems with translations, and drafts of translated data sets or other translated content, as well as anything coming up with the Internationalization of PCGen are to be discussed in that group.<br /> <br /> Anybody interested to help out with translations to other languages, but also those who are just interested in the discussion of the matter, please join the group. Those interested in joining our efforts on the Internationalization of PCGen, please make yourself known on that list.<br /> <br /> This page is an attempt to regroup the information discussed on the list.<br /> <br /> There is a JIRA issue to regroup work on i18n: http://jira.pcgen.org/browse/CODE-733<br /> <br /> =Goal=<br /> <br /> The goal is to allow a user that doesn’t speak English to use the program in his native language with the specifics of his locale. This is an ideal goal, as it is not possible to provide all possible localizations with the low number of volunteers. What is needed is to provide the means for people to create localizations, in order to be able to do that, PCGen must be fully internationalized. Once this is done, the number of localization will slowly grow.<br /> <br /> PCGen interface, especially the part that are data set independent, are already internationalized and partially translated. Updating those translations is one of the needed step but not the only one.<br /> Another element is already internationalized, it’s the use of imperial or metric distance. There is also an option to change it.<br /> <br /> Other steps are need to attain internationalization.<br /> <br /> * First the '''installation process''' should be detailed in many languages, even if it’s only expanding a compressed file then using a script to launch the program.<br /> * All program '''UI strings''' should also be translatable, and so are the '''data strings'''. It seems that the non data strings are already translatable.<br /> * The skills should be '''sorted''' in the UI and in the output sheet according to the natural sorting of the language.<br /> * The '''output sheets''' should be in the user’s language too.<br /> * In case the user create it’s own translation, it should be possible to add the language in the option pane without having to compile the program (or worse have to change some code!). From a technical point of view, it also means that the translations should be external to the jar, or that a simple way to add a translation provided.<br /> <br /> * Saved characters are usually reusable with a new version of the program, and the il8n should not change that. (technical: the saved elements should stay the same, should probably be keys of some sort — I have no idea what the pcg files are)<br /> <br /> * It would be better for the '''measure and weights system''' to be initially set to a logical value rather than a set one, it would depend on the first chosen locale. It is also needed to verify that the imperial and metric system option are actually working.<br /> <br /> There is also other elements that would improve usability for the user, mainly that '''help/documentation''' would be available in his language. This is usually big work to be achieve but doesn’t need any change in PCGen.<br /> Another linked part is to be able to handle '''bug report''' in another language. To do that, the project need some people that would translate the bug reports to English before the development team is able to handle the bug reports. <br /> <br /> = Tools for internationalization and localization =<br /> <br /> To facilitate the internationalization and localization, some tools would be welcome. This tools are not necessary for the user, but are a great addition to maintainers.<br /> <br /> For internationalization:<br /> * a way to determine which strings are translatable or not. Some IDE and some code quality library provide this kind of tool.<br /> * a tool to eliminate (or point at) line that are not used any more in the main bundle file (it is quite big and there seems to bit not displayed text)<br /> <br /> For localization,<br /> * If a specific way of storing the data translation is used, a tool to produce and update a more standard file would be needed. In free project, gettext files (POT, PO) are often used. In Java, ResourceBundle are often used. Another standard format is XLIFF. Such a tool would be able to add new needed translation to a file of the standard format, and flag old translations. From the standard format, it would update the translation of the specific format file. This is needed because there is tool to ease and speed up translation, like Virtaal, ResourceBundle editors, or poedit, and they need a standard file format to be usable. <br /> * a tool test to see if some entries are not translated (maybe a page that would provide statistics) it might be more useful if UI and data separated<br /> * keeping glossary for each languages could help (some terms are found at many places)<br /> * A way to detect incorrect token translation (like using Dager instead of Dagger) is welcome. One way to do this is by generating error or warning when loading the translation file. It would be welcome to generate this list of error without having to launch PCGen, it would also be a nice addition to automatic testing.<br /> <br /> =Choices yet to be made=<br /> <br /> == Application Packaging ==<br /> <br /> As of December 2011, the application and translation of some part of the UI are bundled together.<br /> <br /> The question is should the application be bundled this way, or should there be language specific bundles.<br /> <br /> Pro of a all-in-one<br /> * user can change language on the fly<br /> * no need for explanations on how to install a new language set<br /> <br /> Cons<br /> * file could get quite big<br /> <br /> == Genre class names ==<br /> <br /> In some languages, the class names can have a genre, like the feminine of priest is priestess in English (even if Cleric is the name of the class, it’s just to illustrate the point). Do we want to use those names in the class names or do we stick to whatever name the class is?<br /> <br /> = The goals in more details for PCGen =<br /> <br /> == Default language after clean install ==<br /> <br /> The language should be the system one. To the user, if doesn’t understand the default language, it will be a pain for him to find the option to change the language.<br /> It may be even better if the user is prompted with a choice at the start of the program (or at install) in order to simplify this process. It might be displayed only if the system language doesn’t exist in PCGen.<br /> As noted before, the measure and weight system used need to be chosen by the user at the first install with the default choice depending on the language/locale selected. After first launch or install, the option is changeable in the option preference panel.<br /> <br /> After clean install (no preference files):<br /> # PCGen determine the system language<br /> # if translation exist (and are believed good enough) switch to that language, if not, display an element to ask the user for the language to use.<br /> <br /> == Published language priority ==<br /> <br /> * The data that is not translated in a language, for example books not yet translated, should stay in the language it first get out.<br /> <br /> Books are almost never instantly translated by companies and some aren’t even translated. PCGen is also short on staff. All that means that the translations will probably be coming far after the original one.<br /> To an user that only use translated material, it will not be a problem.<br /> To an user that mix translated material, and not translated one, it shouldn’t become a problem.<br /> <br /> == Using different data languages ==<br /> <br /> * The data should be the same that the English one to avoid having to duplicate data correction.<br /> <br /> Having data in several languages could be a way to do il8n<br /> <br /> Cons<br /> * might need language specific bundles<br /> * a character used with one language cannot be seen by someone with a different language set<br /> * multiple corrections needed when correcting a bug (and if automatic generation of the translation is done, why not do it on the fly)<br /> <br /> It probably should be avoided to do it that way.<br /> <br /> == Separating data and UI translations ==<br /> <br /> * The interface language and the data language should be separate to allow people with book in a different language than their own to use the program more simply.<br /> To the user that speak only one language, it is no advantage nor a problem.<br /> For a user which use book that are not in the language he is most comfortable with, he would put the UI and the data to different languages. Same for someone who doesn’t use translation of the data but prefer to have a program in a language he is more comfortable with.<br /> <br /> == Custom content and localization ==<br /> <br /> * It should be possible to use another language in a data to allow non English speaker to develop custom content in another language. That seem to be a problem when combined with separate language from data. A way to avoid it is to provide easy reference to create custom content.<br /> <br /> This might be a hard to make real combined with other points.<br /> <br /> (might be wrong on this point [[User:Masaru20100|Masaru20100]]) At the moment, when creating custom content, the English token are used. For a non English-speaker, it is a problem, that’s why, ideally, he should be able to create custom by using his own language.<br /> <br /> The problem this cause is that if the user change the language of the program, the custom content either doesn’t work anymore, or all possible translations are checked which would cause massive slowness in the software.<br /> <br /> As it seams undoable, the user should be provided with way to easily find the needed English token for his custom content.<br /> <br /> == Indicating what is translated ==<br /> <br /> It would be good to provide the user with visual information of how much a dataset is translated in his language. It is not really a needed feature but could help him. If he knew that all is not translated, he would be less surprised.<br /> <br /> =Technical Aspects=<br /> <br /> == Language choice UI ==<br /> <br /> There is already options to change the language and the weights and measure system used.<br /> <br /> It might be best to have a drop down list rather than radio choices. New translation should appear in the list.<br /> <br /> == Formatting Issue ==<br /> <br /> Martijn pointed that number should be translatable. In Japanese, the classic number system is almost always used especially in RPG. In fact, translating numbers is not needed, what is needed is formatting numbers.<br /> <br /> That means that 10000 gets formatted 10,000 in English, 10 000 in French. In Japanese, it would either be 10,000 or 1万. There is even a locale to obtain 一万 if needed.<br /> <br /> In Java, this is usually done by using the NumberFormat class with the appropriate locale.<br /> <br /> == Sorting Issue ==<br /> <br /> Sorting is usually language dependent, meaning that the list of skills for example is not sorted the same way in different languages, even if the name are the same. The differences are usually minor between roman letter using languages.<br /> <br /> Actually, the international guys brought up the sorting thing again - another project would be to look into the GUI/core and replacing all the sorting of items with internationalized methods of sorting (Collators).<br /> <br /> Technically, Java already provide ways of sorting based on locale, it’s the Collator classes.<br /> <br /> == Unique Identifier ==<br /> <br /> Apparently there is already unique identifiers in use for items, so the idea would be to have a file with that id as the key and the translation as the value.<br /> <br /> == Output Sheet and Data Language ==<br /> <br /> The question is: should output sheets be locale dependent?<br /> If it is, when a sheet is corrected for one locale, it will need to be edited in all languages. Having languages taken into account in the same sheet make it more complicated.<br /> <br /> === Gender, race and level example ===<br /> <br /> There is also other problem as illustrated by the gender, race and level.<br /> <br /> In English, the gender propose Male, Female, Neuter, Unknown.<br /> In a English statblock, it is usually a line like: Gender Race Level N, as “Female drow cleric 3” for a drow noble [http://www.d20pfsrd.com/bestiary/monster-listings/humanoids/drow-common/drow-noble]. As of this writing, the software output “Female Drow Noble Cleric3”, or “Female Drow Noble Cleric 3” instead in most statblock sheets.<br /> The Unknown gender, and maybe the neuter one too, should probably not output as is in statblock sheets, while not <br /> On my system, with the preferences to use the system language, the female translation is used instead.<br /> <br /> In French, it is Mâle, Femelle, Neuter, Inconnu. The same noble drow is “Drow, prêtre 3” (prêtre is the translation of cleric) [http://www.pathfinder-fr.org/Wiki/Pathfinder-RPG.Drow%20noble.ashx]. I remember reading “Drow (f), prêtre 3”, where the sex is included by using a single letter in parenthesis. It could also have been “Drow, guerrière 3”, where guerrière is the feminine of fighter.<br /> As you can remark, the order is different.<br /> <br /> In Japanese, it looks like “ドラウ（女性）の貴族の3レベル・クレリック” [http://www29.atwiki.jp/prdj/pages/306.html#drow].<br /> <br /> === data translation and output sheet values locale ===<br /> <br /> To write<br /> <br /> == Tom’s technical proposal ==<br /> <br /> So looking back at the earlier thread, I never actually came back with my suggestion, so here is my perspective on localization and how it should be done. This has been made easier by some changes we have in the 6.x line, and having new UI (with what I believe to be a lot more isolation of code that does display of items) is a huge boost here to making this practical.<br /> <br /> First a few base facts as background:<br /> # For [almost] every item (Class, Skill, etc.) there is a unique identifier (generally referred to as a Key - note that if the &quot;KEY&quot; token is not used, then the Key is the name (first entry on the line in the data) (There is an exception to this we'll call problem #1)<br /> # There are basically 4 types of things that need translation:<br /> ## (2a) item names<br /> ## (2b) constants (like spell schools)<br /> ## (2c) variables (like meters/feet/etc.)<br /> ## (2d) Strings (like descriptions)<br /> ## [if anyone can think of more, let me know]<br /> # Most (but not all) tokens are &quot;unique&quot; or otherwise &quot;addressable&quot; in that they can only occur once per object. (Anyone notice that the tokens have started to be specific in the test code &amp; docs about whether they overwrite, add, etc. - this is one reason why - and yes, I've been slowly trying to make progress on this even in the 2007-2009 work) [The exception to this is problem #2)<br /> <br /> A few principles about l10n:<br /> # We must not make producing a data set materially more complicated than it is today (no requirement to put %L10NNAME% type gunk into data)<br /> # We should target an ability to tell us if l10n is complete for any given data set<br /> <br /> With that:<br /> * Following from #1, almost everything we have in the data today is &quot;addressable&quot;. By that, I mean that the OUTPUTNAME for a Skill called &quot;FooBar&quot; can be uniquely called out. The name hierarchy is something like: SKILL//Foo//OUTPUTNAME (exceptions are problems #1,2 to be addressed later)<br /> <br /> Given that, we can actually set principle #1 to &quot;The data remains unchanged&quot; (again, except for problem #2). The entire data set is produced (assume US English for a moment). We then have a unique file for l10n that has things like:<br /> * SKILL:FooBar|Oobarf-ay<br /> * SKILL:FooBar:OUTPUTNAME|ooBarOutF-ay<br /> * etc.<br /> <br /> This (to the first order) covers 2a, 2d.<br /> <br /> For 2b items, we simply have to expand the list of items (SKILL, SPELL) that we are familiar with, so we get things like:<br /> * SPELLSCHOOL:Divination|Ivinationd-ay<br /> <br /> 2c is a bit more complicated, but I can't believe it's all that bad given it's a thing many applications already do (and it's a known thing)<br /> <br /> Each file could be named in such a way that it identifies it's l10n, e.g.:<br /> * srd_de_ch.l10n<br /> * ...and probably placed in a L10N subfolder of the initial dataset.<br /> * (Note I'd recommend we be clear on where these go, AND ALSO require that NO PCC FILES (I'd recommend we say no PCC or LST, but the formal limit would be no PCC) are recognized if they are in the L10N folder... so that the initial directory parse [which is probably one of the slower parts of our boot process] can immediately ignore the l10n directory and not have to look through the file list looking for .PCC files... alternately, we have a l10n folder that is parallel to the data folder, but that then adds complication in needing new preferences to point at multiple l10n folders and requires more complicated structure within that l10n structure to identify WHICH files in that folder map to which datasets... because we can't simply say a given English word will always translate the same way - English is way too overloaded for that.)<br /> <br /> I would recommend we keep things in a small subset of files, and not try to do 1:1 for each data file (that would produce a lot of file sprawl) - but that's not my call.<br /> <br /> Addressing principle #2:<br /> * Since we have a set of items we know would need to be covered (names, certain tokens), we should be able to load those into memory, and load the l10n file into memory and compare. This should be able to produce warnings of 2 kinds:<br /> * (W1) Errors where the base data set contains things that are not translated<br /> * (W2) Errors where the translation file attempts to translate things not in the base data set<br /> * I can't imagine that utility is all that hard to write (just would need to make the list of todos)<br /> <br /> This brings us to problems #1 and #2:<br /> <br /> Problem #1: Non-unique names<br /> * We glossed over this in 6.x, but the truth is that Spell names are not unique. Some of the *RDs have duplicate names (not all, but I forget which). Same is true for languages.<br /> * (1a) Spells can theoretically be differentiated by evaluating the &quot;TYPE&quot; token for Divine, Arcane, or Psionic (those are &quot;magical&quot; items in our code.<br /> * (1b) Languages can theoretically be differentiated between &quot;Spoken&quot; and &quot;Written&quot; as those are &quot;magical&quot; types.<br /> I believe both of those forms of magic are things on our backlog of FREQs to clean up... and the reason they are on the cleanup list really for L10N (as much as it is to just cleanup the overuse of TYPE)<br /> <br /> Problem #2: Non-unique tokens<br /> * There are only a few tokens that are not unique. DESC is one of them, if I recall correctly. The probably solution here is to simply give an identifier to each token. This might require a change to LST Something like:<br /> * DESC*Overall:x<br /> * DESC*Second:x<br /> * (Note: I'm not sure this syntax works or is by any means &quot;good&quot;. DESC:OVERALL|x might be better as it avoids potential issues with using * as a reserved character - take this is a principle of what would have to happen to DESC token, not as a full-blown proposal)<br /> <br /> Note that this naming of each DESC item (And other reusable items), while it breaks the &quot;can't change the data&quot; principle actually helps as much as it hurts... it would give us the ability to do things like:<br /> * DESC:.CLEARID.Overall<br /> * or things similar to that, which is actually a neat benefit for the small overhead of pain it puts into datasets. (Which, by the way, could be converted to whatever we decide on anyway with our nifty converter, so this doesn't seem all that bad)<br /> <br /> So in my mind, the question really is: Is going slightly more than half way good enough? There are areas where we could do translation, and some areas where we need some pretty material core code changes to support it.<br /> <br /> == Translation files ==<br /> <br /> Hi Tom,<br /> <br /> Welcome BACK!!!<br /> <br /> I don't mind the solutions put forth. Unique Identifiers for DESC is no more absurd then the Unique<br /> Identifiers we'll be needing to use more than one CHOOSER on a single line. Plus, the ability to<br /> clear off a Section of DESC would be awesome.<br /> <br /> I'd vote for as complete a job as possible.<br /> <br /> Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> thpr wrote:<br /> <br /> &gt; Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> Idea #1 (bad idea IMHO) is to put it in a separate directory:<br /> data/srd/...<br /> l10n/srd/...<br /> <br /> The problem with that is that you get into synchronization issues... so any time files/directory names change in one place they have to change in another. That's a contract on the data developer. (It would also require multiple l10n directories, since we support multiple data directories, so it's a bunch of code)<br /> <br /> Idea #2: Implicit Subdirectories:<br /> data/srd/*.lst<br /> data/srd/l10n/*.l10n<br /> <br /> That ensures that the l10n directory is associated with the dataset.<br /> <br /> Idea #3: Explicit designation:<br /> data/srd/srd.pcc contains LOCALIZATION:l10n/srd.l10n<br /> then:<br /> data/l10n/srd.l10n<br /> <br /> (would support more than one .l10n file)<br /> <br /> <br /> --- In pcgen_international@yahoogroups.com, Martijn Verburg &lt;martijnverburg@...&gt; wrote:<br /> &gt; &gt; (2) We should target an ability to tell us if l10n is complete for any<br /> &gt; &gt; given data set<br /> &gt; &gt;<br /> &gt; Not sure what you mean by this?<br /> <br /> If there is an object called &quot;Dagger&quot; we need to ensure the file contains:<br /> EQUIPMENT:Dagger|Aggerd-ay<br /> <br /> If it contains:<br /> EQUIPMENT:Dgager|Aggerd-ay<br /> <br /> That is also an error (just like the &quot;unconstructed reference&quot; items are errors. By capturing both type 1 and type 2 error (things that aren't translated as well as things that shouldn't have been translated) we are capturing the vast majority of the simple problems.<br /> <br /> == See also ==<br /> <br /> * [[Localization]]</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Explanation_of_the_Code_Base&diff=3140 2012-09-09T08:12:24Z

<p>Masaru20100: /* pcgen.gui */</p> <hr /> <div>{| align=&quot;right&quot;<br /> | __TOC__<br /> |}<br /> <br /> =High level overview of PCGen Code Base=<br /> <br /> ''src/java/pcgen'' is where the code tree begins. The directories match the package names — so ''pcgen.core'' would be ''src/java/pcgen/core'' (or ''src\java\pcgen\core'' if you are on Windows).<br /> <br /> =Packages=<br /> <br /> ==pcgen.base.*==<br /> <br /> These classes are 'base' library elements. None of these should be objects that are dependent upon pcgen.cdom or pcgen.core classes. The intent is that pcgen.base elements are usable in any project, not specifically PCGen. No item which is deeply tied to the concept of a Role-Playing Game or building a Player Character should be present in this package.<br /> <br /> ==pcgen.cdom.*==<br /> <br /> The pcgen.cdom package contains the core contents of the CDOM structure. There are a number of key subpackages of pcgen.cdom:<br /> <br /> # pcgen.cdom.base: This is the &quot;base&quot; package, which has many key components that the entire PCGen code base is dependent upon. It should be expected that the members of this package have few dependencies outside of pcgen.base* and pcgen.cdom.base*, as this is &quot;low&quot; in the package hierarchy.<br /> # pcgen.cdom.choiceset: This package contains [[CDOM Primitive Choice Set Concept Document|Primitive Choice Sets]]<br /> # pcgen.cdom.content: This package contains the non-CDOM Object content objects that would make up a Player Character. Long term, these objects would be &quot;granted&quot; by CDOM Objects, and thus would appear directly in the CDOM Graph that defines the Player Character (in 5.16, there is very little that distinguishes content from a helper)<br /> # pcgen.cdom.enumeration: This package contains type safe enumerations (such as enums)<br /> # pcgen.cdom.formula: This package contains items related to formula calculations performed by PCGen.<br /> # pcgen.cdom.helper: This package is for &quot;helper&quot; objects of the CDOM Objects found within pcgen.cdom.inst. Generally a &quot;helper&quot; would be contained within the CDOM Object.<br /> # pcgen.cdom.inst: This package is for instances of CDOM Objects. Long term, this will contain many of the objects currently in pcgen.core<br /> # pcgen.cdom.list: This package contains list objects, such as Class Spell List objects. These list objects serve as identifiers for collections of CDOM Objects that are used at runtime.<br /> # pcgen.cdom.reference: This package contains [[CDOM References Concept Document|CDOM References]]<br /> # pcgen.cdom.modifier: This package contains modifiers, a specialized form of content that is designed to alter the behavior of content that is part of a Player Character (The override from a Hit Die Lock is a good example of a modifier)<br /> # pcgen.cdom.util: These are utility packages for classes contained within the pcgen.cdom* packages.<br /> <br /> ==pcgen.rules.*==<br /> <br /> This package contains the classes related to the &quot;Rules Data Store&quot;. The classes here are responsible for loading (and writing) the rules files (PCC, LST), and when combined with the token plugins, this package converts the persistent objects into the instances, lists, content, helpers, references, and other objects of the pcgen.cdom* package.<br /> <br /> ==pcgen.core==<br /> <br /> This is where the business logic goes. This is the main engine that makes things work. The most important class in here is PObject, many classes are subclassed from this (like PCClass, Race, Feat, Spell, etc.). Another important class is PlayerCharacter which is where the player characters are constructed and manipulated. Globals contains the lists of objects loaded when a user loads their desired sources. Constants holds most of the code constants so you do not need to remember their values.<br /> <br /> ==pcgen.gui==<br /> <br /> This is the GUI logic. The main method is in pcGenGUI, and the frame is in PCGen_Frame1. The editors are all defined in the editor package, and all the tab frames are in the tabs package.<br /> This is deprecated in favor of pcgen.gui2 classes.<br /> <br /> ==pcgen.gui2==<br /> <br /> This is the NewGUI logic.<br /> <br /> ==pcgen.io==<br /> <br /> This is where the disk writing stuff occurs, like creating/parsing pcg files, and exporting character sheets.<br /> <br /> ==pcgen.persistence==<br /> <br /> This is where the lst files are read in. The class that controls the loading is SystemLoader.<br /> <br /> ==pcgen.util==<br /> <br /> This is where generally useful classes that are don't fit any of the other categories go.<br /> <br /> ==plugin==<br /> <br /> This is where the pluggable features go. This includes tools such as Character Sheet, Dicebag, Random Name Generator, Encounter Generator, Network Management, Experience Tracker, Initiative Tracker, GM Notes, Overland Travel and the Character Tracker. It also includes processors for the variety of Lst Tokens and Output Tokens. There is also an [[Arch. Discuss of Plugin System|Architectural Discussion of the PCGen Persistence Plugin System]] to help provide additional explanation of how the Lst Tokens work.<br /> <br /> See the Javadocs for further details of the PCGen code base.</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Localization&diff=3132 2012-08-25T03:31:23Z

<p>Masaru20100: /* My Process */</p> <hr /> <div>Localization (l8n) is the process of adapting software for its use in a particular location. It can only be done if the software has been [[internationnalized|Internationnalization]] (i10n). It is mainly translation.<br /> <br /> In the case of PCGen, as with other free software, localization can only be done with the help of the community.<br /> <br /> PCGen both need translation for its user interface (UI) and its data. The data can not be translated for now (as of August 25th 2012), because the internationnalization for the data is not yet done. On top of it, PCGen data is mostly non-free, so providing translation has legal repercutions.<br /> <br /> The translation of the UI is simpler. There is a file for each language, and basically you need to translate all the messages. Beware that, as of August 25th 2012, there is many unused strings in the files.<br /> <br /> (eventually write more for beginners in l8n, provide links?)<br /> <br /> == My Process ==<br /> <br /> By [[User:Masaru20100|Masaru20100]] 03:44, 25 August 2012 (MIST)<br /> <br /> <br /> PCGen uses Java’s ResourceBundle, and use cryptic text as the key of strings (unlike PO that indicate to use ASCII text as keys). It means you need to see two text files at the same time to view original files. Do not edit a copy of another language file, because you’d have a file with a mix of the target language and the copied language, which make it difficult to see what is not yet translated (and also harder to merge). I don’t like the ResourceBundle format because it does not allow me to keep the information that my translations on some strings might not be good. That’s why I keep my main translation in po files and not in bundles.<br /> <br /> Another reason to keep separated files, is that only ASCII characters can be included in a ResourceBundle, other characters need to be unicode escaped (things like \u0145).<br /> <br /> Each time I update my svn copy, I create a new POT file from the default resource bundle. Then I update my latest po file of the language I’m working on, update it with the POT file and save it with a new name (using the svn version number). I then proceed to edit the po file. I save the file, update the resource bundle in the code directory then (I think its needed to see changes) build pcgen.<br /> <br /> To update, I just do<br /> <br /> # svn up<br /> <br /> To create the new POT, I use prop2po which is part of the package translate-toolkit on Debian. I have a little script that I usually call with the svn number.<br /> <br /> # newpot 17045<br /> <br /> ''newpot''<br /> <br /> #!/bin/zsh<br /> <br /> # argument is name of version<br /> &lt;nowiki&gt;if [[ -z $1 ]];&lt;/nowiki&gt;<br /> then<br /> echo Argument needed<br /> exit 1<br /> else<br /> name=.$1<br /> fi<br /> <br /> # Create the new POT from the Language bundle<br /> prop2po ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle.properties LanguageBundle$name.pot<br /> #po2xliff -P LanguageBundle$name.pot LanguageBundle$name.xlf<br /> <br /> I use virtaal to open the latest po file, update it with the POT then save it with the new version number (poedit would work fine but doesn’t show the comments in the po file). I then edit the file.<br /> <br /> Once my change are done, or I want to view them, I use a script to update the resource bundle file. You’ll have to change the name of the language you’d work with there (fr ja en_GB in my case). It uses po2prop also part of the translate-toolkit package.<br /> <br /> # updatebundle 17045<br /> <br /> ''updatebundle''<br /> <br /> #!/bin/zsh<br /> <br /> # argument is name of version<br /> if &lt;nowiki&gt;[[ -z $1 ]];&lt;/nowiki&gt;<br /> then<br /> echo Argument needed<br /> exit 1<br /> fi<br /> <br /> # Create the new POT from the Language bundle<br /> for lang in fr ja en_GB<br /> do<br /> &lt;nowiki&gt;if [[ -f LanguageBundle.$1.$lang.po ]];&lt;/nowiki&gt;<br /> then<br /> po2prop --fuzzy --errorlevel=traceback -t ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle.properties LanguageBundle.$1.$lang.po ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle_$lang.properties<br /> fi<br /> done<br /> <br /> I hope those scripts help some one, you’ll probably need to change them (to fix the path for example).</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Localization&diff=3131 2012-08-25T02:29:56Z

<p>Masaru20100: Added part on encoding of bundles, nowiki tag for scripts.</p> <hr /> <div>Localization (l8n) is the process of adapting software for its use in a particular location. It can only be done if the software has been [[internationnalized|Internationnalization]] (i10n). It is mainly translation.<br /> <br /> In the case of PCGen, as with other free software, localization can only be done with the help of the community.<br /> <br /> PCGen both need translation for its user interface (UI) and its data. The data can not be translated for now (as of August 25th 2012), because the internationnalization for the data is not yet done. On top of it, PCGen data is mostly non-free, so providing translation has legal repercutions.<br /> <br /> The translation of the UI is simpler. There is a file for each language, and basically you need to translate all the messages. Beware that, as of August 25th 2012, there is many unused strings in the files.<br /> <br /> (eventually write more for beginners in l8n, provide links?)<br /> <br /> == My Process ==<br /> <br /> By [[User:Masaru20100|Masaru20100]] 03:44, 25 August 2012 (MIST)<br /> <br /> <br /> PCGen uses Java’s ResourceBundle, and use cryptic text as the key of strings (unlike PO that indicate to use ASCII text as keys). It means you need to see two text files at the same time to view original files. Do not edit a copy of another language file, because you’d have a file with a mix of the target language and the copied language, which make it difficult to see what is not yet translated (and also harder to merge). I don’t like the ResourceBundle format because it does not allow me to keep the information that my translations on some strings might not be good. That’s why I keep my main translation in po files and not in bundles.<br /> <br /> Another reason to keep separated files, is that only ASCII characters can be included in a ResourceBundle, other characters need to be unicode escaped (things like \u0145).<br /> <br /> Each time I update my svn copy, I create a new POT file from the default resource bundle. Then I update my latest po file of the language I’m working on, update it with the POT file and save it with a new name (using the svn version number). I then proceed to edit the po file. I save the file, update the resource bundle in the code directory then (I think its needed to see changes) build pcgen.<br /> <br /> To update, I just do<br /> <br /> # svn up<br /> <br /> To create the new POT, I use prop2po which is part of the package translate-toolkit on Debian. I have a little script that I usually call with the svn number.<br /> <br /> # newpot 17045<br /> <br /> ''newpot''<br /> &lt;nowiki&gt;<br /> #!/bin/zsh<br /> <br /> # argument is name of version<br /> if [[ -z $1 ]];<br /> then<br /> echo Argument needed<br /> exit 1<br /> else<br /> name=.$1<br /> fi<br /> <br /> # Create the new POT from the Language bundle<br /> prop2po ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle.properties LanguageBundle$name.pot<br /> #po2xliff -P LanguageBundle$name.pot LanguageBundle$name.xlf&lt;/nowiki&gt;<br /> <br /> I use virtaal to open the latest po file, update it with the POT then save it with the new version number (poedit would work fine but doesn’t show the comments in the po file). I then edit the file.<br /> <br /> Once my change are done, or I want to view them, I use a script to update the resource bundle file. You’ll have to change the name of the language you’d work with there (fr ja en_GB in my case). It uses po2prop also part of the translate-toolkit package.<br /> <br /> # updatebundle 17045<br /> <br /> ''updatebundle''<br /> &lt;nowiki&gt;<br /> #!/bin/zsh<br /> <br /> # argument is name of version<br /> if [[ -z $1 ]];<br /> then<br /> echo Argument needed<br /> exit 1<br /> fi<br /> <br /> # Create the new POT from the Language bundle<br /> for lang in fr ja en_GB<br /> do<br /> if [[ -f LanguageBundle.$1.$lang.po ]];<br /> then<br /> po2prop --fuzzy --errorlevel=traceback -t ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle.properties LanguageBundle.$1.$lang.po ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle_$lang.properties<br /> fi<br /> done<br /> &lt;/nowiki&gt;<br /> <br /> I hope those scripts help some one, you’ll probably need to change them (to fix the path for example).</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Internationalization&diff=3130 2012-08-24T16:46:30Z

<p>Masaru20100: added link to Localization</p> <hr /> <div>=Introduction=<br /> <br /> PCGen for 6.0+ releases will be attempting Internationalization (probably not in 6.0). Internalization is the process of changing software to allow it to be usable in any language and any locale. It is often abbreviated i18n. It is needed in PCGen because the software was not thought from the ground up to be usable from a language other than English. Localization is the process of creating the translations for a particular language, to do so the software need to able to handle internationalization. To deal with those, a Yahoo group called [http://groups.yahoo.com/group/pcgen_international pcgen_international] has been created.<br /> <br /> Requests for translations, problems with translations, and drafts of translated data sets or other translated content, as well as anything coming up with the Internationalization of PCGen are to be discussed in that group.<br /> <br /> Anybody interested to help out with translations to other languages, but also those who are just interested in the discussion of the matter, please join the group. Those interested in joining our efforts on the Internationalization of PCGen, please make yourself known on that list.<br /> <br /> This page is an attempt to regroup the information discussed on the list.<br /> <br /> There is a JIRA issue to regroup work on i18n: http://jira.pcgen.org/browse/CODE-733<br /> <br /> =Goal=<br /> <br /> The goal is to allow a user that doesn’t speak English to use the program in his native language with the specifics of his locale. This is an ideal goal, as it is not possible to provide all possible localizations with the low number of volunteers. What is needed is to provide the means for people to create localizations, in order to be able to do that, PCGen must be fully internationalized. Once this is done, the number of localization will slowly grow.<br /> <br /> PCGen interface, especially the part that are data set independent, are already internationalized and partially translated. Updating those translations is one of the needed step but not the only one.<br /> Another element is already internationalized, it’s the use of imperial or metric distance. There is also an option to change it.<br /> <br /> Other steps are need to attain internationalization.<br /> <br /> * First the '''installation process''' should be detailed in many languages, even if it’s only expanding a compressed file then using a script to launch the program.<br /> * All program '''UI strings''' should also be translatable, and so are the '''data strings'''. It seems that the non data strings are already translatable.<br /> * The skills should be '''sorted''' in the UI and in the output sheet according to the natural sorting of the language.<br /> * The '''output sheets''' should be in the user’s language too.<br /> * In case the user create it’s own translation, it should be possible to add the language in the option pane without having to compile the program (or worse have to change some code!). From a technical point of view, it also means that the translations should be external to the jar, or that a simple way to add a translation provided.<br /> <br /> * Saved characters are usually reusable with a new version of the program, and the il8n should not change that. (technical: the saved elements should stay the same, should probably be keys of some sort — I have no idea what the pcg files are)<br /> <br /> * It would be better for the '''measure and weights system''' to be initially set to a logical value rather than a set one, it would depend on the first chosen locale. It is also needed to verify that the imperial and metric system option are actually working.<br /> <br /> There is also other elements that would improve usability for the user, mainly that '''help/documentation''' would be available in his language. This is usually big work to be achieve but doesn’t need any change in PCGen.<br /> Another linked part is to be able to handle '''bug report''' in another language. To do that, the project need some people that would translate the bug reports to English before the development team is able to handle the bug reports. <br /> <br /> = Tools for internationalization and localization =<br /> <br /> To facilitate the internationalization and localization, some tools would be welcome. This tools are not necessary for the user, but are a great addition to maintainers.<br /> <br /> For internationalization:<br /> * a way to determine which strings are translatable or not. Some IDE and some code quality library provide this kind of tool.<br /> * a tool to eliminate (or point at) line that are not used any more in the main bundle file (it is quite big and there seems to bit not displayed text)<br /> <br /> For localization,<br /> * If a specific way of storing the data translation is used, a tool to produce and update a more standard file would be needed. In free project, gettext files (POT, PO) are often used. In Java, ResourceBundle are often used. Another standard format is XLIFF. Such a tool would be able to add new needed translation to a file of the standard format, and flag old translations. From the standard format, it would update the translation of the specific format file. This is needed because there is tool to ease and speed up translation, like Virtaal, ResourceBundle editors, or poedit, and they need a standard file format to be usable. <br /> * a tool test to see if some entries are not translated (maybe a page that would provide statistics) it might be more useful if UI and data separated<br /> * keeping glossary for each languages could help (some terms are found at many places)<br /> * A way to detect incorrect token translation (like using Dager instead of Dagger) is welcome. One way to do this is by generating error or warning when loading the translation file. It would be welcome to generate this list of error without having to launch PCGen, it would also be a nice addition to automatic testing.<br /> <br /> =Choices yet to be made=<br /> <br /> == Application Packaging ==<br /> <br /> As of December 2011, the application and translation of some part of the UI are bundled together.<br /> <br /> The question is should the application be bundled this way, or should there be language specific bundles.<br /> <br /> Pro of a all-in-one<br /> * user can change language on the fly<br /> * no need for explanations on how to install a new language set<br /> <br /> Cons<br /> * file could get quite big<br /> <br /> == Genre class names ==<br /> <br /> In some languages, the class names can have a genre, like the feminine of priest is priestess in English (even if Cleric is the name of the class, it’s just to illustrate the point). Do we want to use those names in the class names or do we stick to whatever name the class is?<br /> <br /> = The goals in more details for PCGen =<br /> <br /> == Default language after clean install ==<br /> <br /> The language should be the system one. To the user, if doesn’t understand the default language, it will be a pain for him to find the option to change the language.<br /> It may be even better if the user is prompted with a choice at the start of the program (or at install) in order to simplify this process. It might be displayed only if the system language doesn’t exist in PCGen.<br /> As noted before, the measure and weight system used need to be chosen by the user at the first install with the default choice depending on the language/locale selected. After first launch or install, the option is changeable in the option preference panel.<br /> <br /> After clean install (no preference files):<br /> # PCGen determine the system language<br /> # if translation exist (and are believed good enough) switch to that language, if not, display an element to ask the user for the language to use.<br /> <br /> == Published language priority ==<br /> <br /> * The data that is not translated in a language, for example books not yet translated, should stay in the language it first get out.<br /> <br /> Books are almost never instantly translated by companies and some aren’t even translated. PCGen is also short on staff. All that means that the translations will probably be coming far after the original one.<br /> To an user that only use translated material, it will not be a problem.<br /> To an user that mix translated material, and not translated one, it shouldn’t become a problem.<br /> <br /> == Using different data languages ==<br /> <br /> * The data should be the same that the English one to avoid having to duplicate data correction.<br /> <br /> Having data in several languages could be a way to do il8n<br /> <br /> Cons<br /> * might need language specific bundles<br /> * a character used with one language cannot be seen by someone with a different language set<br /> * multiple corrections needed when correcting a bug (and if automatic generation of the translation is done, why not do it on the fly)<br /> <br /> It probably should be avoided to do it that way.<br /> <br /> == Separating data and UI translations ==<br /> <br /> * The interface language and the data language should be separate to allow people with book in a different language than their own to use the program more simply.<br /> To the user that speak only one language, it is no advantage nor a problem.<br /> For a user which use book that are not in the language he is most comfortable with, he would put the UI and the data to different languages. Same for someone who doesn’t use translation of the data but prefer to have a program in a language he is more comfortable with.<br /> <br /> == Custom content and localization ==<br /> <br /> * It should be possible to use another language in a data to allow non English speaker to develop custom content in another language. That seem to be a problem when combined with separate language from data. A way to avoid it is to provide easy reference to create custom content.<br /> <br /> This might be a hard to make real combined with other points.<br /> <br /> (might be wrong on this point [[User:Masaru20100|Masaru20100]]) At the moment, when creating custom content, the English token are used. For a non English-speaker, it is a problem, that’s why, ideally, he should be able to create custom by using his own language.<br /> <br /> The problem this cause is that if the user change the language of the program, the custom content either doesn’t work anymore, or all possible translations are checked which would cause massive slowness in the software.<br /> <br /> As it seams undoable, the user should be provided with way to easily find the needed English token for his custom content.<br /> <br /> == Indicating what is translated ==<br /> <br /> It would be good to provide the user with visual information of how much a dataset is translated in his language. It is not really a needed feature but could help him. If he knew that all is not translated, he would be less surprised.<br /> <br /> =Technical Aspects=<br /> <br /> == Language choice UI ==<br /> <br /> There is already options to change the language and the weights and measure system used.<br /> <br /> It might be best to have a drop down list rather than radio choices. New translation should appear in the list.<br /> <br /> == Formatting Issue ==<br /> <br /> Martijn pointed that number should be translatable. In Japanese, the classic number system is almost always used especially in RPG. In fact, translating numbers is not needed, what is needed is formatting numbers.<br /> <br /> That means that 10000 gets formatted 10,000 in English, 10 000 in French. In Japanese, it would either be 10,000 or 1万. There is even a locale to obtain 一万 if needed.<br /> <br /> In Java, this is usually done by using the NumberFormat class with the appropriate locale.<br /> <br /> == Sorting Issue ==<br /> <br /> Sorting is usually language dependent, meaning that the list of skills for example is not sorted the same way in different languages, even if the name are the same. The differences are usually minor between roman letter using languages.<br /> <br /> Actually, the international guys brought up the sorting thing again - another project would be to look into the GUI/core and replacing all the sorting of items with internationalized methods of sorting (Collators).<br /> <br /> Technically, Java already provide ways of sorting based on locale, it’s the Collator classes.<br /> <br /> == Unique Identifier ==<br /> <br /> Apparently there is already unique identifiers in use for items, so the idea would be to have a file with that id as the key and the translation as the value.<br /> <br /> == Tom’s technical proposal ==<br /> <br /> So looking back at the earlier thread, I never actually came back with my suggestion, so here is my perspective on localization and how it should be done. This has been made easier by some changes we have in the 6.x line, and having new UI (with what I believe to be a lot more isolation of code that does display of items) is a huge boost here to making this practical.<br /> <br /> First a few base facts as background:<br /> # For [almost] every item (Class, Skill, etc.) there is a unique identifier (generally referred to as a Key - note that if the &quot;KEY&quot; token is not used, then the Key is the name (first entry on the line in the data) (There is an exception to this we'll call problem #1)<br /> # There are basically 4 types of things that need translation:<br /> ## (2a) item names<br /> ## (2b) constants (like spell schools)<br /> ## (2c) variables (like meters/feet/etc.)<br /> ## (2d) Strings (like descriptions)<br /> ## [if anyone can think of more, let me know]<br /> # Most (but not all) tokens are &quot;unique&quot; or otherwise &quot;addressable&quot; in that they can only occur once per object. (Anyone notice that the tokens have started to be specific in the test code &amp; docs about whether they overwrite, add, etc. - this is one reason why - and yes, I've been slowly trying to make progress on this even in the 2007-2009 work) [The exception to this is problem #2)<br /> <br /> A few principles about l10n:<br /> # We must not make producing a data set materially more complicated than it is today (no requirement to put %L10NNAME% type gunk into data)<br /> # We should target an ability to tell us if l10n is complete for any given data set<br /> <br /> With that:<br /> * Following from #1, almost everything we have in the data today is &quot;addressable&quot;. By that, I mean that the OUTPUTNAME for a Skill called &quot;FooBar&quot; can be uniquely called out. The name hierarchy is something like: SKILL//Foo//OUTPUTNAME (exceptions are problems #1,2 to be addressed later)<br /> <br /> Given that, we can actually set principle #1 to &quot;The data remains unchanged&quot; (again, except for problem #2). The entire data set is produced (assume US English for a moment). We then have a unique file for l10n that has things like:<br /> * SKILL:FooBar|Oobarf-ay<br /> * SKILL:FooBar:OUTPUTNAME|ooBarOutF-ay<br /> * etc.<br /> <br /> This (to the first order) covers 2a, 2d.<br /> <br /> For 2b items, we simply have to expand the list of items (SKILL, SPELL) that we are familiar with, so we get things like:<br /> * SPELLSCHOOL:Divination|Ivinationd-ay<br /> <br /> 2c is a bit more complicated, but I can't believe it's all that bad given it's a thing many applications already do (and it's a known thing)<br /> <br /> Each file could be named in such a way that it identifies it's l10n, e.g.:<br /> * srd_de_ch.l10n<br /> * ...and probably placed in a L10N subfolder of the initial dataset.<br /> * (Note I'd recommend we be clear on where these go, AND ALSO require that NO PCC FILES (I'd recommend we say no PCC or LST, but the formal limit would be no PCC) are recognized if they are in the L10N folder... so that the initial directory parse [which is probably one of the slower parts of our boot process] can immediately ignore the l10n directory and not have to look through the file list looking for .PCC files... alternately, we have a l10n folder that is parallel to the data folder, but that then adds complication in needing new preferences to point at multiple l10n folders and requires more complicated structure within that l10n structure to identify WHICH files in that folder map to which datasets... because we can't simply say a given English word will always translate the same way - English is way too overloaded for that.)<br /> <br /> I would recommend we keep things in a small subset of files, and not try to do 1:1 for each data file (that would produce a lot of file sprawl) - but that's not my call.<br /> <br /> Addressing principle #2:<br /> * Since we have a set of items we know would need to be covered (names, certain tokens), we should be able to load those into memory, and load the l10n file into memory and compare. This should be able to produce warnings of 2 kinds:<br /> * (W1) Errors where the base data set contains things that are not translated<br /> * (W2) Errors where the translation file attempts to translate things not in the base data set<br /> * I can't imagine that utility is all that hard to write (just would need to make the list of todos)<br /> <br /> This brings us to problems #1 and #2:<br /> <br /> Problem #1: Non-unique names<br /> * We glossed over this in 6.x, but the truth is that Spell names are not unique. Some of the *RDs have duplicate names (not all, but I forget which). Same is true for languages.<br /> * (1a) Spells can theoretically be differentiated by evaluating the &quot;TYPE&quot; token for Divine, Arcane, or Psionic (those are &quot;magical&quot; items in our code.<br /> * (1b) Languages can theoretically be differentiated between &quot;Spoken&quot; and &quot;Written&quot; as those are &quot;magical&quot; types.<br /> I believe both of those forms of magic are things on our backlog of FREQs to clean up... and the reason they are on the cleanup list really for L10N (as much as it is to just cleanup the overuse of TYPE)<br /> <br /> Problem #2: Non-unique tokens<br /> * There are only a few tokens that are not unique. DESC is one of them, if I recall correctly. The probably solution here is to simply give an identifier to each token. This might require a change to LST Something like:<br /> * DESC*Overall:x<br /> * DESC*Second:x<br /> * (Note: I'm not sure this syntax works or is by any means &quot;good&quot;. DESC:OVERALL|x might be better as it avoids potential issues with using * as a reserved character - take this is a principle of what would have to happen to DESC token, not as a full-blown proposal)<br /> <br /> Note that this naming of each DESC item (And other reusable items), while it breaks the &quot;can't change the data&quot; principle actually helps as much as it hurts... it would give us the ability to do things like:<br /> * DESC:.CLEARID.Overall<br /> * or things similar to that, which is actually a neat benefit for the small overhead of pain it puts into datasets. (Which, by the way, could be converted to whatever we decide on anyway with our nifty converter, so this doesn't seem all that bad)<br /> <br /> So in my mind, the question really is: Is going slightly more than half way good enough? There are areas where we could do translation, and some areas where we need some pretty material core code changes to support it.<br /> <br /> == Translation files ==<br /> <br /> Hi Tom,<br /> <br /> Welcome BACK!!!<br /> <br /> I don't mind the solutions put forth. Unique Identifiers for DESC is no more absurd then the Unique<br /> Identifiers we'll be needing to use more than one CHOOSER on a single line. Plus, the ability to<br /> clear off a Section of DESC would be awesome.<br /> <br /> I'd vote for as complete a job as possible.<br /> <br /> Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> thpr wrote:<br /> <br /> &gt; Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> Idea #1 (bad idea IMHO) is to put it in a separate directory:<br /> data/srd/...<br /> l10n/srd/...<br /> <br /> The problem with that is that you get into synchronization issues... so any time files/directory names change in one place they have to change in another. That's a contract on the data developer. (It would also require multiple l10n directories, since we support multiple data directories, so it's a bunch of code)<br /> <br /> Idea #2: Implicit Subdirectories:<br /> data/srd/*.lst<br /> data/srd/l10n/*.l10n<br /> <br /> That ensures that the l10n directory is associated with the dataset.<br /> <br /> Idea #3: Explicit designation:<br /> data/srd/srd.pcc contains LOCALIZATION:l10n/srd.l10n<br /> then:<br /> data/l10n/srd.l10n<br /> <br /> (would support more than one .l10n file)<br /> <br /> <br /> --- In pcgen_international@yahoogroups.com, Martijn Verburg &lt;martijnverburg@...&gt; wrote:<br /> &gt; &gt; (2) We should target an ability to tell us if l10n is complete for any<br /> &gt; &gt; given data set<br /> &gt; &gt;<br /> &gt; Not sure what you mean by this?<br /> <br /> If there is an object called &quot;Dagger&quot; we need to ensure the file contains:<br /> EQUIPMENT:Dagger|Aggerd-ay<br /> <br /> If it contains:<br /> EQUIPMENT:Dgager|Aggerd-ay<br /> <br /> That is also an error (just like the &quot;unconstructed reference&quot; items are errors. By capturing both type 1 and type 2 error (things that aren't translated as well as things that shouldn't have been translated) we are capturing the vast majority of the simple problems.<br /> <br /> == See also ==<br /> <br /> * [[Localization]]</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Localization&diff=3129 2012-08-24T16:44:45Z

<p>Masaru20100: /* My Process */</p> <hr /> <div>Localization (l8n) is the process of adapting software for its use in a particular location. It can only be done if the software has been [[internationnalized|Internationnalisation]] (i10n). It is mainly translation.<br /> <br /> In the case of PCGen, as with other free software, localization can only be done with the help of the community.<br /> <br /> PCGen both need translation for its user interface (UI) and its data. The data can not be translated for now (as of August 25th 2012), because the internationnalization for the data is not yet done. On top of it, PCGen data is mostly non-free, so providing translation has legal repercutions.<br /> <br /> The translation of the UI is simpler. There is a file for each language, and basically you need to translate all the messages. Beware that, as of August 25th 2012, there is many unused strings in the files.<br /> <br /> (eventually write more for beginners in l8n, provide links?)<br /> <br /> == My Process ==<br /> <br /> By [[User:Masaru20100|Masaru20100]] 03:44, 25 August 2012 (MIST)<br /> <br /> <br /> PCGen uses Java’s ResourceBundle, and use cryptic text as the key of strings (unlike PO that indicate to use ASCII text as keys). It means you need to see two text files at the same time to view original files. Do not edit a copy of another language file, because you’d have a file with a mix of the target language and the copied language, which make it difficult to see what is not yet translated (and also harder to merge). I don’t like the ResourceBundle format because it does not allow me to keep the information that my translations on some strings might not be good. That’s why I keep my main translation in po files and not in bundles.<br /> <br /> Each time I update my svn copy, I create a new POT file from the default resource bundle. Then I update my latest po file of the language I’m working on, update it with the POT file and save it with a new name (using the svn version number). I then proceed to edit the po file. I save the file, update the resource bundle in the code directory then (I think its needed to see changes) build pcgen.<br /> <br /> To update, I just do<br /> <br /> # svn up<br /> <br /> To create the new POT, I use prop2po which is part of the package translate-toolkit on Debian. I have a little script that I usually call with the svn number.<br /> <br /> # newpot 17045<br /> <br /> ''newpot''<br /> <br /> #!/bin/zsh<br /> <br /> # argument is name of version<br /> if [[ -z $1 ]];<br /> then<br /> echo Argument needed<br /> exit 1<br /> else<br /> name=.$1<br /> fi<br /> <br /> # Create the new POT from the Language bundle<br /> prop2po ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle.properties LanguageBundle$name.pot<br /> #po2xliff -P LanguageBundle$name.pot LanguageBundle$name.xlf<br /> <br /> I use virtaal to open the latest po file, update it with the POT then save it with the new version number (poedit would work fine but doesn’t show the comments in the po file). I then edit the file.<br /> <br /> Once my change are done, or I want to view them, I use a script to update the resource bundle file. You’ll have to change the name of the language you’d work with there (fr ja en_GB in my case). It uses po2prop also part of the translate-toolkit package.<br /> <br /> # updatebundle 17045<br /> <br /> ''updatebundle''<br /> <br /> #!/bin/zsh<br /> <br /> # argument is name of version<br /> if [[ -z $1 ]];<br /> then<br /> echo Argument needed<br /> exit 1<br /> fi<br /> <br /> # Create the new POT from the Language bundle<br /> for lang in fr ja en_GB<br /> do<br /> if [[ -f LanguageBundle.$1.$lang.po ]];<br /> then<br /> po2prop --fuzzy --errorlevel=traceback -t ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle.properties LanguageBundle.$1.$lang.po ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle_$lang.properties<br /> fi<br /> done<br /> <br /> I hope those scripts help some one, you’ll probably need to change them (to fix the path for example).</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Localization&diff=3128 2012-08-24T16:44:11Z

<p>Masaru20100: /* My Process */</p> <hr /> <div>Localization (l8n) is the process of adapting software for its use in a particular location. It can only be done if the software has been [[internationnalized|Internationnalisation]] (i10n). It is mainly translation.<br /> <br /> In the case of PCGen, as with other free software, localization can only be done with the help of the community.<br /> <br /> PCGen both need translation for its user interface (UI) and its data. The data can not be translated for now (as of August 25th 2012), because the internationnalization for the data is not yet done. On top of it, PCGen data is mostly non-free, so providing translation has legal repercutions.<br /> <br /> The translation of the UI is simpler. There is a file for each language, and basically you need to translate all the messages. Beware that, as of August 25th 2012, there is many unused strings in the files.<br /> <br /> (eventually write more for beginners in l8n, provide links?)<br /> <br /> == My Process ==<br /> <br /> PCGen uses Java’s ResourceBundle, and use cryptic text as the key of strings (unlike PO that indicate to use ASCII text as keys). It means you need to see two text files at the same time to view original files. Do not edit a copy of another language file, because you’d have a file with a mix of the target language and the copied language, which make it difficult to see what is not yet translated (and also harder to merge). I don’t like the ResourceBundle format because it does not allow me to keep the information that my translations on some strings might not be good. That’s why I keep my main translation in po files and not in bundles.<br /> <br /> Each time I update my svn copy, I create a new POT file from the default resource bundle. Then I update my latest po file of the language I’m working on, update it with the POT file and save it with a new name (using the svn version number). I then proceed to edit the po file. I save the file, update the resource bundle in the code directory then (I think its needed to see changes) build pcgen.<br /> <br /> To update, I just do<br /> <br /> # svn up<br /> <br /> To create the new POT, I use prop2po which is part of the package translate-toolkit on Debian. I have a little script that I usually call with the svn number.<br /> <br /> # newpot 17045<br /> <br /> ''newpot''<br /> <br /> #!/bin/zsh<br /> <br /> # argument is name of version<br /> if [[ -z $1 ]];<br /> then<br /> echo Argument needed<br /> exit 1<br /> else<br /> name=.$1<br /> fi<br /> <br /> # Create the new POT from the Language bundle<br /> prop2po ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle.properties LanguageBundle$name.pot<br /> #po2xliff -P LanguageBundle$name.pot LanguageBundle$name.xlf<br /> <br /> I use virtaal to open the latest po file, update it with the POT then save it with the new version number (poedit would work fine but doesn’t show the comments in the po file). I then edit the file.<br /> <br /> Once my change are done, or I want to view them, I use a script to update the resource bundle file. You’ll have to change the name of the language you’d work with there (fr ja en_GB in my case). It uses po2prop also part of the translate-toolkit package.<br /> <br /> # updatebundle 17045<br /> <br /> ''updatebundle''<br /> <br /> #!/bin/zsh<br /> <br /> # argument is name of version<br /> if [[ -z $1 ]];<br /> then<br /> echo Argument needed<br /> exit 1<br /> fi<br /> <br /> # Create the new POT from the Language bundle<br /> for lang in fr ja en_GB<br /> do<br /> if [[ -f LanguageBundle.$1.$lang.po ]];<br /> then<br /> po2prop --fuzzy --errorlevel=traceback -t ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle.properties LanguageBundle.$1.$lang.po ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle_$lang.properties<br /> fi<br /> done<br /> <br /> I hope those scripts help some one, you’ll probably need to change them (to fix the path for example).<br /> <br /> [[User:Masaru20100|Masaru20100]] 03:44, 25 August 2012 (MIST)</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Localization&diff=3127 2012-08-24T16:43:47Z

<p>Masaru20100: New page on l8n</p> <hr /> <div>Localization (l8n) is the process of adapting software for its use in a particular location. It can only be done if the software has been [[internationnalized|Internationnalisation]] (i10n). It is mainly translation.<br /> <br /> In the case of PCGen, as with other free software, localization can only be done with the help of the community.<br /> <br /> PCGen both need translation for its user interface (UI) and its data. The data can not be translated for now (as of August 25th 2012), because the internationnalization for the data is not yet done. On top of it, PCGen data is mostly non-free, so providing translation has legal repercutions.<br /> <br /> The translation of the UI is simpler. There is a file for each language, and basically you need to translate all the messages. Beware that, as of August 25th 2012, there is many unused strings in the files.<br /> <br /> (eventually write more for beginners in l8n, provide links?)<br /> <br /> == My Process ==<br /> <br /> PCGen uses Java’s ResourceBundle, and use cryptic text as the key of strings (unlike PO that indicate to use ASCII text as keys). It means you need to see two text files at the same time to view original files. Do not edit a copy of another language file, because you’d have a file with a mix of the target language and the copied language, which make it difficult to see what is not yet translated (and also harder to merge). I don’t like the ResourceBundle format because it does not allow me to keep the information that my translations on some strings might not be good. That’s why I keep my main translation in po files and not in bundles.<br /> <br /> Each time I update my svn copy, I create a new POT file from the default resource bundle. Then I update my latest po file of the language I’m working on, update it with the POT file and save it with a new name (using the svn version number). I then proceed to edit the po file. I save the file, update the resource bundle in the code directory then (I think its needed to see changes) build pcgen.<br /> <br /> To update, I just do<br /> <br /> # svn up<br /> <br /> To create the new POT, I use prop2po which is part of the package translate-toolkit on Debian. I have a little script that I usually call with the svn number.<br /> <br /> # newpot 17045<br /> <br /> ''newpot''<br /> <br /> #!/bin/zsh<br /> <br /> # argument is name of version<br /> if [[ -z $1 ]];<br /> then<br /> echo Argument needed<br /> exit 1<br /> else<br /> name=.$1<br /> fi<br /> <br /> # Create the new POT from the Language bundle<br /> prop2po ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle.properties LanguageBundle$name.pot<br /> #po2xliff -P LanguageBundle$name.pot LanguageBundle$name.xlf<br /> <br /> I use virtaal to open the latest po file, update it with the POT then save it with the new version number (poedit would work fine but doesn’t show the comments in the po file). I then edit the file.<br /> <br /> Once my change are done, or I want to view them, I use a script to update the resource bundle file. You’ll have to change the name of the language you’d work with there (fr ja en_GB in my case). It uses po2prop also part of the translate-toolkit package.<br /> <br /> # updatebundle 17045<br /> <br /> ''updatebundle''<br /> <br /> #!/bin/zsh<br /> <br /> # argument is name of version<br /> if [[ -z $1 ]];<br /> then<br /> echo Argument needed<br /> exit 1<br /> fi<br /> <br /> # Create the new POT from the Language bundle<br /> for lang in fr ja en_GB<br /> do<br /> if [[ -f LanguageBundle.$1.$lang.po ]];<br /> then<br /> po2prop --fuzzy --errorlevel=traceback -t ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle.properties LanguageBundle.$1.$lang.po ../pcgen-svn/code/src/java/pcgen/resources/lang/LanguageBundle_$lang.properties<br /> fi<br /> done<br /> <br /> I hope those scripts help some one, you’ll probably need to change them (to fix the path for example).</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Joining_the_Code_Team&diff=3026 2012-02-16T01:53:26Z

<p>Masaru20100: /* Your first change */ Changed external link to internal one</p> <hr /> <div>{| align=&quot;right&quot;<br /> | __TOC__<br /> |}<br /> <br /> =Introduction=<br /> <br /> A list of what to do to join the code team<br /> <br /> =General steps=<br /> <br /> # Get a [http://groups.yahoo.com Yahoo Groups] id.<br /> # Be sure to subscribe to the [http://groups.yahoo.com/group/pcgen_developers/ Developer's mailing list]<br /> # Introduce yourself on the list<br /> # Find the [http://games.groups.yahoo.com/group/pcgen/ pcgen] project on Yahoo! Groups and join.<br /> # Get as many IM IDs as possible (most people use Pidgin, Meebo or Gaim) and get online with them.<br /> # Also join the IRC channel #pcgen on ~DALnet &amp; ~Freenode, we have our meetings and group discussions there in real time.<br /> # Get the latest official version of PCGen from SF and run it.<br /> # Get an id at [http://sourceforge.net sourceforge].<br /> # Find the [http://sourceforge.net/projects/pcgen pcgen] project.<br /> <br /> =Your first change=<br /> # Get your [[Basic Developer Setup|development environment set up]]<br /> # Select an issue that interests you, either from our issues flagged as bitseize in the [http://jira.pcgen.org/secure/IssueNavigator.jspa?reset=true&amp;mode=hide&amp;jqlQuery=component+%3D+Bitesize+AND+project+%3D+CODE+AND+resolution+%3D+Unresolved+ORDER+BY+updated+DESC Code] and [http://jira.pcgen.org/secure/IssueNavigator.jspa?reset=true&amp;mode=hide&amp;jqlQuery=component+%3D+Bitesize+AND+project+%3D+NEWTAG+AND+resolution+%3D+Unresolved+ORDER+BY+updated+DESC New Tag] projects, or one you have encountered.<br /> # Build a fix that fixes the bug or implements the new feature<br /> # Test the fix, preferably by writing a unit test<br /> # Send the fix as a patch to the developers list asking for a review.<br /> # Your patch will be reviewed and friendly feedback provided by one of the team.<br /> <br /> =Joining the team=<br /> <br /> # Once our patch is accepted, the Code Silverback will add you the project on Sourceforge.<br /> # Your first check-in will be your fix<br /> # Your second should be adding your name to the team listing!<br /> # Put your details in this wiki under the [[Code]] team!<br /> <br /> Any questions, don't hesitate to ask!</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Internationalization&diff=2964 2011-12-19T02:37:12Z

<p>Masaru20100: /* Introduction */ added jira link</p> <hr /> <div>=Introduction=<br /> <br /> PCGen for 6.0+ releases will be attempting Internationalization (probably not in 6.0). Internalization is the process of changing software to allow it to be usable in any language and any locale. It is often abbreviated i18n. It is needed in PCGen because the software was not thought from the ground up to be usable from a language other than English. Localization is the process of creating the translations for a particular language, to do so the software need to able to handle internationalization. To deal with those, a Yahoo group called [http://groups.yahoo.com/group/pcgen_international pcgen_international] has been created.<br /> <br /> Requests for translations, problems with translations, and drafts of translated data sets or other translated content, as well as anything coming up with the Internationalization of PCGen are to be discussed in that group.<br /> <br /> Anybody interested to help out with translations to other languages, but also those who are just interested in the discussion of the matter, please join the group. Those interested in joining our efforts on the Internationalization of PCGen, please make yourself known on that list.<br /> <br /> This page is an attempt to regroup the information discussed on the list.<br /> <br /> There is a JIRA issue to regroup work on i18n: http://jira.pcgen.org/browse/CODE-733<br /> <br /> =Goal=<br /> <br /> The goal is to allow a user that doesn’t speak English to use the program in his native language with the specifics of his locale. This is an ideal goal, as it is not possible to provide all possible localizations with the low number of volunteers. What is needed is to provide the means for people to create localizations, in order to be able to do that, PCGen must be fully internationalized. Once this is done, the number of localization will slowly grow.<br /> <br /> PCGen interface, especially the part that are data set independent, are already internationalized and partially translated. Updating those translations is one of the needed step but not the only one.<br /> Another element is already internationalized, it’s the use of imperial or metric distance. There is also an option to change it.<br /> <br /> Other steps are need to attain internationalization.<br /> <br /> * First the '''installation process''' should be detailed in many languages, even if it’s only expanding a compressed file then using a script to launch the program.<br /> * All program '''UI strings''' should also be translatable, and so are the '''data strings'''. It seems that the non data strings are already translatable.<br /> * The skills should be '''sorted''' in the UI and in the output sheet according to the natural sorting of the language.<br /> * The '''output sheets''' should be in the user’s language too.<br /> * In case the user create it’s own translation, it should be possible to add the language in the option pane without having to compile the program (or worse have to change some code!). From a technical point of view, it also means that the translations should be external to the jar, or that a simple way to add a translation provided.<br /> <br /> * Saved characters are usually reusable with a new version of the program, and the il8n should not change that. (technical: the saved elements should stay the same, should probably be keys of some sort — I have no idea what the pcg files are)<br /> <br /> * It would be better for the '''measure and weights system''' to be initially set to a logical value rather than a set one, it would depend on the first chosen locale. It is also needed to verify that the imperial and metric system option are actually working.<br /> <br /> There is also other elements that would improve usability for the user, mainly that '''help/documentation''' would be available in his language. This is usually big work to be achieve but doesn’t need any change in PCGen.<br /> Another linked part is to be able to handle '''bug report''' in another language. To do that, the project need some people that would translate the bug reports to English before the development team is able to handle the bug reports. <br /> <br /> = Tools for internationalization and localization =<br /> <br /> To facilitate the internationalization and localization, some tools would be welcome. This tools are not necessary for the user, but are a great addition to maintainers.<br /> <br /> For internationalization:<br /> * a way to determine which strings are translatable or not. Some IDE and some code quality library provide this kind of tool.<br /> * a tool to eliminate (or point at) line that are not used any more in the main bundle file (it is quite big and there seems to bit not displayed text)<br /> <br /> For localization,<br /> * If a specific way of storing the data translation is used, a tool to produce and update a more standard file would be needed. In free project, gettext files (POT, PO) are often used. In Java, ResourceBundle are often used. Another standard format is XLIFF. Such a tool would be able to add new needed translation to a file of the standard format, and flag old translations. From the standard format, it would update the translation of the specific format file. This is needed because there is tool to ease and speed up translation, like Virtaal, ResourceBundle editors, or poedit, and they need a standard file format to be usable. <br /> * a tool test to see if some entries are not translated (maybe a page that would provide statistics) it might be more useful if UI and data separated<br /> * keeping glossary for each languages could help (some terms are found at many places)<br /> * A way to detect incorrect token translation (like using Dager instead of Dagger) is welcome. One way to do this is by generating error or warning when loading the translation file. It would be welcome to generate this list of error without having to launch PCGen, it would also be a nice addition to automatic testing.<br /> <br /> =Choices yet to be made=<br /> <br /> == Application Packaging ==<br /> <br /> As of December 2011, the application and translation of some part of the UI are bundled together.<br /> <br /> The question is should the application be bundled this way, or should there be language specific bundles.<br /> <br /> Pro of a all-in-one<br /> * user can change language on the fly<br /> * no need for explanations on how to install a new language set<br /> <br /> Cons<br /> * file could get quite big<br /> <br /> == Genre class names ==<br /> <br /> In some languages, the class names can have a genre, like the feminine of priest is priestess in English (even if Cleric is the name of the class, it’s just to illustrate the point). Do we want to use those names in the class names or do we stick to whatever name the class is?<br /> <br /> = The goals in more details for PCGen =<br /> <br /> == Default language after clean install ==<br /> <br /> The language should be the system one. To the user, if doesn’t understand the default language, it will be a pain for him to find the option to change the language.<br /> It may be even better if the user is prompted with a choice at the start of the program (or at install) in order to simplify this process. It might be displayed only if the system language doesn’t exist in PCGen.<br /> As noted before, the measure and weight system used need to be chosen by the user at the first install with the default choice depending on the language/locale selected. After first launch or install, the option is changeable in the option preference panel.<br /> <br /> After clean install (no preference files):<br /> # PCGen determine the system language<br /> # if translation exist (and are believed good enough) switch to that language, if not, display an element to ask the user for the language to use.<br /> <br /> == Published language priority ==<br /> <br /> * The data that is not translated in a language, for example books not yet translated, should stay in the language it first get out.<br /> <br /> Books are almost never instantly translated by companies and some aren’t even translated. PCGen is also short on staff. All that means that the translations will probably be coming far after the original one.<br /> To an user that only use translated material, it will not be a problem.<br /> To an user that mix translated material, and not translated one, it shouldn’t become a problem.<br /> <br /> == Using different data languages ==<br /> <br /> * The data should be the same that the English one to avoid having to duplicate data correction.<br /> <br /> Having data in several languages could be a way to do il8n<br /> <br /> Cons<br /> * might need language specific bundles<br /> * a character used with one language cannot be seen by someone with a different language set<br /> * multiple corrections needed when correcting a bug (and if automatic generation of the translation is done, why not do it on the fly)<br /> <br /> It probably should be avoided to do it that way.<br /> <br /> == Separating data and UI translations ==<br /> <br /> * The interface language and the data language should be separate to allow people with book in a different language than their own to use the program more simply.<br /> To the user that speak only one language, it is no advantage nor a problem.<br /> For a user which use book that are not in the language he is most comfortable with, he would put the UI and the data to different languages. Same for someone who doesn’t use translation of the data but prefer to have a program in a language he is more comfortable with.<br /> <br /> == Custom content and localization ==<br /> <br /> * It should be possible to use another language in a data to allow non English speaker to develop custom content in another language. That seem to be a problem when combined with separate language from data. A way to avoid it is to provide easy reference to create custom content.<br /> <br /> This might be a hard to make real combined with other points.<br /> <br /> (might be wrong on this point [[User:Masaru20100|Masaru20100]]) At the moment, when creating custom content, the English token are used. For a non English-speaker, it is a problem, that’s why, ideally, he should be able to create custom by using his own language.<br /> <br /> The problem this cause is that if the user change the language of the program, the custom content either doesn’t work anymore, or all possible translations are checked which would cause massive slowness in the software.<br /> <br /> As it seams undoable, the user should be provided with way to easily find the needed English token for his custom content.<br /> <br /> == Indicating what is translated ==<br /> <br /> It would be good to provide the user with visual information of how much a dataset is translated in his language. It is not really a needed feature but could help him. If he knew that all is not translated, he would be less surprised.<br /> <br /> =Technical Aspects=<br /> <br /> == Language choice UI ==<br /> <br /> There is already options to change the language and the weights and measure system used.<br /> <br /> It might be best to have a drop down list rather than radio choices. New translation should appear in the list.<br /> <br /> == Formatting Issue ==<br /> <br /> Martijn pointed that number should be translatable. In Japanese, the classic number system is almost always used especially in RPG. In fact, translating numbers is not needed, what is needed is formatting numbers.<br /> <br /> That means that 10000 gets formatted 10,000 in English, 10 000 in French. In Japanese, it would either be 10,000 or 1万. There is even a locale to obtain 一万 if needed.<br /> <br /> In Java, this is usually done by using the NumberFormat class with the appropriate locale.<br /> <br /> == Sorting Issue ==<br /> <br /> Sorting is usually language dependent, meaning that the list of skills for example is not sorted the same way in different languages, even if the name are the same. The differences are usually minor between roman letter using languages.<br /> <br /> Actually, the international guys brought up the sorting thing again - another project would be to look into the GUI/core and replacing all the sorting of items with internationalized methods of sorting (Collators).<br /> <br /> Technically, Java already provide ways of sorting based on locale, it’s the Collator classes.<br /> <br /> == Unique Identifier ==<br /> <br /> Apparently there is already unique identifiers in use for items, so the idea would be to have a file with that id as the key and the translation as the value.<br /> <br /> == Tom’s technical proposal ==<br /> <br /> So looking back at the earlier thread, I never actually came back with my suggestion, so here is my perspective on localization and how it should be done. This has been made easier by some changes we have in the 6.x line, and having new UI (with what I believe to be a lot more isolation of code that does display of items) is a huge boost here to making this practical.<br /> <br /> First a few base facts as background:<br /> # For [almost] every item (Class, Skill, etc.) there is a unique identifier (generally referred to as a Key - note that if the &quot;KEY&quot; token is not used, then the Key is the name (first entry on the line in the data) (There is an exception to this we'll call problem #1)<br /> # There are basically 4 types of things that need translation:<br /> ## (2a) item names<br /> ## (2b) constants (like spell schools)<br /> ## (2c) variables (like meters/feet/etc.)<br /> ## (2d) Strings (like descriptions)<br /> ## [if anyone can think of more, let me know]<br /> # Most (but not all) tokens are &quot;unique&quot; or otherwise &quot;addressable&quot; in that they can only occur once per object. (Anyone notice that the tokens have started to be specific in the test code &amp; docs about whether they overwrite, add, etc. - this is one reason why - and yes, I've been slowly trying to make progress on this even in the 2007-2009 work) [The exception to this is problem #2)<br /> <br /> A few principles about l10n:<br /> # We must not make producing a data set materially more complicated than it is today (no requirement to put %L10NNAME% type gunk into data)<br /> # We should target an ability to tell us if l10n is complete for any given data set<br /> <br /> With that:<br /> * Following from #1, almost everything we have in the data today is &quot;addressable&quot;. By that, I mean that the OUTPUTNAME for a Skill called &quot;FooBar&quot; can be uniquely called out. The name hierarchy is something like: SKILL//Foo//OUTPUTNAME (exceptions are problems #1,2 to be addressed later)<br /> <br /> Given that, we can actually set principle #1 to &quot;The data remains unchanged&quot; (again, except for problem #2). The entire data set is produced (assume US English for a moment). We then have a unique file for l10n that has things like:<br /> * SKILL:FooBar|Oobarf-ay<br /> * SKILL:FooBar:OUTPUTNAME|ooBarOutF-ay<br /> * etc.<br /> <br /> This (to the first order) covers 2a, 2d.<br /> <br /> For 2b items, we simply have to expand the list of items (SKILL, SPELL) that we are familiar with, so we get things like:<br /> * SPELLSCHOOL:Divination|Ivinationd-ay<br /> <br /> 2c is a bit more complicated, but I can't believe it's all that bad given it's a thing many applications already do (and it's a known thing)<br /> <br /> Each file could be named in such a way that it identifies it's l10n, e.g.:<br /> * srd_de_ch.l10n<br /> * ...and probably placed in a L10N subfolder of the initial dataset.<br /> * (Note I'd recommend we be clear on where these go, AND ALSO require that NO PCC FILES (I'd recommend we say no PCC or LST, but the formal limit would be no PCC) are recognized if they are in the L10N folder... so that the initial directory parse [which is probably one of the slower parts of our boot process] can immediately ignore the l10n directory and not have to look through the file list looking for .PCC files... alternately, we have a l10n folder that is parallel to the data folder, but that then adds complication in needing new preferences to point at multiple l10n folders and requires more complicated structure within that l10n structure to identify WHICH files in that folder map to which datasets... because we can't simply say a given English word will always translate the same way - English is way too overloaded for that.)<br /> <br /> I would recommend we keep things in a small subset of files, and not try to do 1:1 for each data file (that would produce a lot of file sprawl) - but that's not my call.<br /> <br /> Addressing principle #2:<br /> * Since we have a set of items we know would need to be covered (names, certain tokens), we should be able to load those into memory, and load the l10n file into memory and compare. This should be able to produce warnings of 2 kinds:<br /> * (W1) Errors where the base data set contains things that are not translated<br /> * (W2) Errors where the translation file attempts to translate things not in the base data set<br /> * I can't imagine that utility is all that hard to write (just would need to make the list of todos)<br /> <br /> This brings us to problems #1 and #2:<br /> <br /> Problem #1: Non-unique names<br /> * We glossed over this in 6.x, but the truth is that Spell names are not unique. Some of the *RDs have duplicate names (not all, but I forget which). Same is true for languages.<br /> * (1a) Spells can theoretically be differentiated by evaluating the &quot;TYPE&quot; token for Divine, Arcane, or Psionic (those are &quot;magical&quot; items in our code.<br /> * (1b) Languages can theoretically be differentiated between &quot;Spoken&quot; and &quot;Written&quot; as those are &quot;magical&quot; types.<br /> I believe both of those forms of magic are things on our backlog of FREQs to clean up... and the reason they are on the cleanup list really for L10N (as much as it is to just cleanup the overuse of TYPE)<br /> <br /> Problem #2: Non-unique tokens<br /> * There are only a few tokens that are not unique. DESC is one of them, if I recall correctly. The probably solution here is to simply give an identifier to each token. This might require a change to LST Something like:<br /> * DESC*Overall:x<br /> * DESC*Second:x<br /> * (Note: I'm not sure this syntax works or is by any means &quot;good&quot;. DESC:OVERALL|x might be better as it avoids potential issues with using * as a reserved character - take this is a principle of what would have to happen to DESC token, not as a full-blown proposal)<br /> <br /> Note that this naming of each DESC item (And other reusable items), while it breaks the &quot;can't change the data&quot; principle actually helps as much as it hurts... it would give us the ability to do things like:<br /> * DESC:.CLEARID.Overall<br /> * or things similar to that, which is actually a neat benefit for the small overhead of pain it puts into datasets. (Which, by the way, could be converted to whatever we decide on anyway with our nifty converter, so this doesn't seem all that bad)<br /> <br /> So in my mind, the question really is: Is going slightly more than half way good enough? There are areas where we could do translation, and some areas where we need some pretty material core code changes to support it.<br /> <br /> == Translation files ==<br /> <br /> Hi Tom,<br /> <br /> Welcome BACK!!!<br /> <br /> I don't mind the solutions put forth. Unique Identifiers for DESC is no more absurd then the Unique<br /> Identifiers we'll be needing to use more than one CHOOSER on a single line. Plus, the ability to<br /> clear off a Section of DESC would be awesome.<br /> <br /> I'd vote for as complete a job as possible.<br /> <br /> Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> thpr wrote:<br /> <br /> &gt; Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> Idea #1 (bad idea IMHO) is to put it in a separate directory:<br /> data/srd/...<br /> l10n/srd/...<br /> <br /> The problem with that is that you get into synchronization issues... so any time files/directory names change in one place they have to change in another. That's a contract on the data developer. (It would also require multiple l10n directories, since we support multiple data directories, so it's a bunch of code)<br /> <br /> Idea #2: Implicit Subdirectories:<br /> data/srd/*.lst<br /> data/srd/l10n/*.l10n<br /> <br /> That ensures that the l10n directory is associated with the dataset.<br /> <br /> Idea #3: Explicit designation:<br /> data/srd/srd.pcc contains LOCALIZATION:l10n/srd.l10n<br /> then:<br /> data/l10n/srd.l10n<br /> <br /> (would support more than one .l10n file)<br /> <br /> <br /> --- In pcgen_international@yahoogroups.com, Martijn Verburg &lt;martijnverburg@...&gt; wrote:<br /> &gt; &gt; (2) We should target an ability to tell us if l10n is complete for any<br /> &gt; &gt; given data set<br /> &gt; &gt;<br /> &gt; Not sure what you mean by this?<br /> <br /> If there is an object called &quot;Dagger&quot; we need to ensure the file contains:<br /> EQUIPMENT:Dagger|Aggerd-ay<br /> <br /> If it contains:<br /> EQUIPMENT:Dgager|Aggerd-ay<br /> <br /> That is also an error (just like the &quot;unconstructed reference&quot; items are errors. By capturing both type 1 and type 2 error (things that aren't translated as well as things that shouldn't have been translated) we are capturing the vast majority of the simple problems.</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Talk:Internationalization&diff=2963 2011-12-19T02:30:25Z

<p>Masaru20100: Blanked the page</p> <hr /> <div></div>

Masaru20100 http://159.203.101.162/w/index.php?title=Internationalization&diff=2962 2011-12-19T02:29:51Z

<p>Masaru20100: Updated from PCGen i18n list</p> <hr /> <div>=Introduction=<br /> <br /> PCGen for 6.0+ releases will be attempting Internationalization (probably not in 6.0). Internalization is the process of changing software to allow it to be usable in any language and any locale. It is often abbreviated i18n. It is needed in PCGen because the software was not thought from the ground up to be usable from a language other than English. Localization is the process of creating the translations for a particular language, to do so the software need to able to handle internationalization. To deal with those, a Yahoo group called [http://groups.yahoo.com/group/pcgen_international pcgen_international] has been created.<br /> <br /> Requests for translations, problems with translations, and drafts of translated data sets or other translated content, as well as anything coming up with the Internationalization of PCGen are to be discussed in that group.<br /> <br /> Anybody interested to help out with translations to other languages, but also those who are just interested in the discussion of the matter, please join the group. Those interested in joining our efforts on the Internationalization of PCGen, please make yourself known on that list.<br /> <br /> This page is an attempt to regroup the information discussed on the list.<br /> <br /> =Goal=<br /> <br /> The goal is to allow a user that doesn’t speak English to use the program in his native language with the specifics of his locale. This is an ideal goal, as it is not possible to provide all possible localizations with the low number of volunteers. What is needed is to provide the means for people to create localizations, in order to be able to do that, PCGen must be fully internationalized. Once this is done, the number of localization will slowly grow.<br /> <br /> PCGen interface, especially the part that are data set independent, are already internationalized and partially translated. Updating those translations is one of the needed step but not the only one.<br /> Another element is already internationalized, it’s the use of imperial or metric distance. There is also an option to change it.<br /> <br /> Other steps are need to attain internationalization.<br /> <br /> * First the '''installation process''' should be detailed in many languages, even if it’s only expanding a compressed file then using a script to launch the program.<br /> * All program '''UI strings''' should also be translatable, and so are the '''data strings'''. It seems that the non data strings are already translatable.<br /> * The skills should be '''sorted''' in the UI and in the output sheet according to the natural sorting of the language.<br /> * The '''output sheets''' should be in the user’s language too.<br /> * In case the user create it’s own translation, it should be possible to add the language in the option pane without having to compile the program (or worse have to change some code!). From a technical point of view, it also means that the translations should be external to the jar, or that a simple way to add a translation provided.<br /> <br /> * Saved characters are usually reusable with a new version of the program, and the il8n should not change that. (technical: the saved elements should stay the same, should probably be keys of some sort — I have no idea what the pcg files are)<br /> <br /> * It would be better for the '''measure and weights system''' to be initially set to a logical value rather than a set one, it would depend on the first chosen locale. It is also needed to verify that the imperial and metric system option are actually working.<br /> <br /> There is also other elements that would improve usability for the user, mainly that '''help/documentation''' would be available in his language. This is usually big work to be achieve but doesn’t need any change in PCGen.<br /> Another linked part is to be able to handle '''bug report''' in another language. To do that, the project need some people that would translate the bug reports to English before the development team is able to handle the bug reports. <br /> <br /> = Tools for internationalization and localization =<br /> <br /> To facilitate the internationalization and localization, some tools would be welcome. This tools are not necessary for the user, but are a great addition to maintainers.<br /> <br /> For internationalization:<br /> * a way to determine which strings are translatable or not. Some IDE and some code quality library provide this kind of tool.<br /> * a tool to eliminate (or point at) line that are not used any more in the main bundle file (it is quite big and there seems to bit not displayed text)<br /> <br /> For localization,<br /> * If a specific way of storing the data translation is used, a tool to produce and update a more standard file would be needed. In free project, gettext files (POT, PO) are often used. In Java, ResourceBundle are often used. Another standard format is XLIFF. Such a tool would be able to add new needed translation to a file of the standard format, and flag old translations. From the standard format, it would update the translation of the specific format file. This is needed because there is tool to ease and speed up translation, like Virtaal, ResourceBundle editors, or poedit, and they need a standard file format to be usable. <br /> * a tool test to see if some entries are not translated (maybe a page that would provide statistics) it might be more useful if UI and data separated<br /> * keeping glossary for each languages could help (some terms are found at many places)<br /> * A way to detect incorrect token translation (like using Dager instead of Dagger) is welcome. One way to do this is by generating error or warning when loading the translation file. It would be welcome to generate this list of error without having to launch PCGen, it would also be a nice addition to automatic testing.<br /> <br /> =Choices yet to be made=<br /> <br /> == Application Packaging ==<br /> <br /> As of December 2011, the application and translation of some part of the UI are bundled together.<br /> <br /> The question is should the application be bundled this way, or should there be language specific bundles.<br /> <br /> Pro of a all-in-one<br /> * user can change language on the fly<br /> * no need for explanations on how to install a new language set<br /> <br /> Cons<br /> * file could get quite big<br /> <br /> == Genre class names ==<br /> <br /> In some languages, the class names can have a genre, like the feminine of priest is priestess in English (even if Cleric is the name of the class, it’s just to illustrate the point). Do we want to use those names in the class names or do we stick to whatever name the class is?<br /> <br /> = The goals in more details for PCGen =<br /> <br /> == Default language after clean install ==<br /> <br /> The language should be the system one. To the user, if doesn’t understand the default language, it will be a pain for him to find the option to change the language.<br /> It may be even better if the user is prompted with a choice at the start of the program (or at install) in order to simplify this process. It might be displayed only if the system language doesn’t exist in PCGen.<br /> As noted before, the measure and weight system used need to be chosen by the user at the first install with the default choice depending on the language/locale selected. After first launch or install, the option is changeable in the option preference panel.<br /> <br /> After clean install (no preference files):<br /> # PCGen determine the system language<br /> # if translation exist (and are believed good enough) switch to that language, if not, display an element to ask the user for the language to use.<br /> <br /> == Published language priority ==<br /> <br /> * The data that is not translated in a language, for example books not yet translated, should stay in the language it first get out.<br /> <br /> Books are almost never instantly translated by companies and some aren’t even translated. PCGen is also short on staff. All that means that the translations will probably be coming far after the original one.<br /> To an user that only use translated material, it will not be a problem.<br /> To an user that mix translated material, and not translated one, it shouldn’t become a problem.<br /> <br /> == Using different data languages ==<br /> <br /> * The data should be the same that the English one to avoid having to duplicate data correction.<br /> <br /> Having data in several languages could be a way to do il8n<br /> <br /> Cons<br /> * might need language specific bundles<br /> * a character used with one language cannot be seen by someone with a different language set<br /> * multiple corrections needed when correcting a bug (and if automatic generation of the translation is done, why not do it on the fly)<br /> <br /> It probably should be avoided to do it that way.<br /> <br /> == Separating data and UI translations ==<br /> <br /> * The interface language and the data language should be separate to allow people with book in a different language than their own to use the program more simply.<br /> To the user that speak only one language, it is no advantage nor a problem.<br /> For a user which use book that are not in the language he is most comfortable with, he would put the UI and the data to different languages. Same for someone who doesn’t use translation of the data but prefer to have a program in a language he is more comfortable with.<br /> <br /> == Custom content and localization ==<br /> <br /> * It should be possible to use another language in a data to allow non English speaker to develop custom content in another language. That seem to be a problem when combined with separate language from data. A way to avoid it is to provide easy reference to create custom content.<br /> <br /> This might be a hard to make real combined with other points.<br /> <br /> (might be wrong on this point [[User:Masaru20100|Masaru20100]]) At the moment, when creating custom content, the English token are used. For a non English-speaker, it is a problem, that’s why, ideally, he should be able to create custom by using his own language.<br /> <br /> The problem this cause is that if the user change the language of the program, the custom content either doesn’t work anymore, or all possible translations are checked which would cause massive slowness in the software.<br /> <br /> As it seams undoable, the user should be provided with way to easily find the needed English token for his custom content.<br /> <br /> == Indicating what is translated ==<br /> <br /> It would be good to provide the user with visual information of how much a dataset is translated in his language. It is not really a needed feature but could help him. If he knew that all is not translated, he would be less surprised.<br /> <br /> =Technical Aspects=<br /> <br /> == Language choice UI ==<br /> <br /> There is already options to change the language and the weights and measure system used.<br /> <br /> It might be best to have a drop down list rather than radio choices. New translation should appear in the list.<br /> <br /> == Formatting Issue ==<br /> <br /> Martijn pointed that number should be translatable. In Japanese, the classic number system is almost always used especially in RPG. In fact, translating numbers is not needed, what is needed is formatting numbers.<br /> <br /> That means that 10000 gets formatted 10,000 in English, 10 000 in French. In Japanese, it would either be 10,000 or 1万. There is even a locale to obtain 一万 if needed.<br /> <br /> In Java, this is usually done by using the NumberFormat class with the appropriate locale.<br /> <br /> == Sorting Issue ==<br /> <br /> Sorting is usually language dependent, meaning that the list of skills for example is not sorted the same way in different languages, even if the name are the same. The differences are usually minor between roman letter using languages.<br /> <br /> Actually, the international guys brought up the sorting thing again - another project would be to look into the GUI/core and replacing all the sorting of items with internationalized methods of sorting (Collators).<br /> <br /> Technically, Java already provide ways of sorting based on locale, it’s the Collator classes.<br /> <br /> == Unique Identifier ==<br /> <br /> Apparently there is already unique identifiers in use for items, so the idea would be to have a file with that id as the key and the translation as the value.<br /> <br /> == Tom’s technical proposal ==<br /> <br /> So looking back at the earlier thread, I never actually came back with my suggestion, so here is my perspective on localization and how it should be done. This has been made easier by some changes we have in the 6.x line, and having new UI (with what I believe to be a lot more isolation of code that does display of items) is a huge boost here to making this practical.<br /> <br /> First a few base facts as background:<br /> # For [almost] every item (Class, Skill, etc.) there is a unique identifier (generally referred to as a Key - note that if the &quot;KEY&quot; token is not used, then the Key is the name (first entry on the line in the data) (There is an exception to this we'll call problem #1)<br /> # There are basically 4 types of things that need translation:<br /> ## (2a) item names<br /> ## (2b) constants (like spell schools)<br /> ## (2c) variables (like meters/feet/etc.)<br /> ## (2d) Strings (like descriptions)<br /> ## [if anyone can think of more, let me know]<br /> # Most (but not all) tokens are &quot;unique&quot; or otherwise &quot;addressable&quot; in that they can only occur once per object. (Anyone notice that the tokens have started to be specific in the test code &amp; docs about whether they overwrite, add, etc. - this is one reason why - and yes, I've been slowly trying to make progress on this even in the 2007-2009 work) [The exception to this is problem #2)<br /> <br /> A few principles about l10n:<br /> # We must not make producing a data set materially more complicated than it is today (no requirement to put %L10NNAME% type gunk into data)<br /> # We should target an ability to tell us if l10n is complete for any given data set<br /> <br /> With that:<br /> * Following from #1, almost everything we have in the data today is &quot;addressable&quot;. By that, I mean that the OUTPUTNAME for a Skill called &quot;FooBar&quot; can be uniquely called out. The name hierarchy is something like: SKILL//Foo//OUTPUTNAME (exceptions are problems #1,2 to be addressed later)<br /> <br /> Given that, we can actually set principle #1 to &quot;The data remains unchanged&quot; (again, except for problem #2). The entire data set is produced (assume US English for a moment). We then have a unique file for l10n that has things like:<br /> * SKILL:FooBar|Oobarf-ay<br /> * SKILL:FooBar:OUTPUTNAME|ooBarOutF-ay<br /> * etc.<br /> <br /> This (to the first order) covers 2a, 2d.<br /> <br /> For 2b items, we simply have to expand the list of items (SKILL, SPELL) that we are familiar with, so we get things like:<br /> * SPELLSCHOOL:Divination|Ivinationd-ay<br /> <br /> 2c is a bit more complicated, but I can't believe it's all that bad given it's a thing many applications already do (and it's a known thing)<br /> <br /> Each file could be named in such a way that it identifies it's l10n, e.g.:<br /> * srd_de_ch.l10n<br /> * ...and probably placed in a L10N subfolder of the initial dataset.<br /> * (Note I'd recommend we be clear on where these go, AND ALSO require that NO PCC FILES (I'd recommend we say no PCC or LST, but the formal limit would be no PCC) are recognized if they are in the L10N folder... so that the initial directory parse [which is probably one of the slower parts of our boot process] can immediately ignore the l10n directory and not have to look through the file list looking for .PCC files... alternately, we have a l10n folder that is parallel to the data folder, but that then adds complication in needing new preferences to point at multiple l10n folders and requires more complicated structure within that l10n structure to identify WHICH files in that folder map to which datasets... because we can't simply say a given English word will always translate the same way - English is way too overloaded for that.)<br /> <br /> I would recommend we keep things in a small subset of files, and not try to do 1:1 for each data file (that would produce a lot of file sprawl) - but that's not my call.<br /> <br /> Addressing principle #2:<br /> * Since we have a set of items we know would need to be covered (names, certain tokens), we should be able to load those into memory, and load the l10n file into memory and compare. This should be able to produce warnings of 2 kinds:<br /> * (W1) Errors where the base data set contains things that are not translated<br /> * (W2) Errors where the translation file attempts to translate things not in the base data set<br /> * I can't imagine that utility is all that hard to write (just would need to make the list of todos)<br /> <br /> This brings us to problems #1 and #2:<br /> <br /> Problem #1: Non-unique names<br /> * We glossed over this in 6.x, but the truth is that Spell names are not unique. Some of the *RDs have duplicate names (not all, but I forget which). Same is true for languages.<br /> * (1a) Spells can theoretically be differentiated by evaluating the &quot;TYPE&quot; token for Divine, Arcane, or Psionic (those are &quot;magical&quot; items in our code.<br /> * (1b) Languages can theoretically be differentiated between &quot;Spoken&quot; and &quot;Written&quot; as those are &quot;magical&quot; types.<br /> I believe both of those forms of magic are things on our backlog of FREQs to clean up... and the reason they are on the cleanup list really for L10N (as much as it is to just cleanup the overuse of TYPE)<br /> <br /> Problem #2: Non-unique tokens<br /> * There are only a few tokens that are not unique. DESC is one of them, if I recall correctly. The probably solution here is to simply give an identifier to each token. This might require a change to LST Something like:<br /> * DESC*Overall:x<br /> * DESC*Second:x<br /> * (Note: I'm not sure this syntax works or is by any means &quot;good&quot;. DESC:OVERALL|x might be better as it avoids potential issues with using * as a reserved character - take this is a principle of what would have to happen to DESC token, not as a full-blown proposal)<br /> <br /> Note that this naming of each DESC item (And other reusable items), while it breaks the &quot;can't change the data&quot; principle actually helps as much as it hurts... it would give us the ability to do things like:<br /> * DESC:.CLEARID.Overall<br /> * or things similar to that, which is actually a neat benefit for the small overhead of pain it puts into datasets. (Which, by the way, could be converted to whatever we decide on anyway with our nifty converter, so this doesn't seem all that bad)<br /> <br /> So in my mind, the question really is: Is going slightly more than half way good enough? There are areas where we could do translation, and some areas where we need some pretty material core code changes to support it.<br /> <br /> == Translation files ==<br /> <br /> Hi Tom,<br /> <br /> Welcome BACK!!!<br /> <br /> I don't mind the solutions put forth. Unique Identifiers for DESC is no more absurd then the Unique<br /> Identifiers we'll be needing to use more than one CHOOSER on a single line. Plus, the ability to<br /> clear off a Section of DESC would be awesome.<br /> <br /> I'd vote for as complete a job as possible.<br /> <br /> Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> thpr wrote:<br /> <br /> &gt; Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> Idea #1 (bad idea IMHO) is to put it in a separate directory:<br /> data/srd/...<br /> l10n/srd/...<br /> <br /> The problem with that is that you get into synchronization issues... so any time files/directory names change in one place they have to change in another. That's a contract on the data developer. (It would also require multiple l10n directories, since we support multiple data directories, so it's a bunch of code)<br /> <br /> Idea #2: Implicit Subdirectories:<br /> data/srd/*.lst<br /> data/srd/l10n/*.l10n<br /> <br /> That ensures that the l10n directory is associated with the dataset.<br /> <br /> Idea #3: Explicit designation:<br /> data/srd/srd.pcc contains LOCALIZATION:l10n/srd.l10n<br /> then:<br /> data/l10n/srd.l10n<br /> <br /> (would support more than one .l10n file)<br /> <br /> <br /> --- In pcgen_international@yahoogroups.com, Martijn Verburg &lt;martijnverburg@...&gt; wrote:<br /> &gt; &gt; (2) We should target an ability to tell us if l10n is complete for any<br /> &gt; &gt; given data set<br /> &gt; &gt;<br /> &gt; Not sure what you mean by this?<br /> <br /> If there is an object called &quot;Dagger&quot; we need to ensure the file contains:<br /> EQUIPMENT:Dagger|Aggerd-ay<br /> <br /> If it contains:<br /> EQUIPMENT:Dgager|Aggerd-ay<br /> <br /> That is also an error (just like the &quot;unconstructed reference&quot; items are errors. By capturing both type 1 and type 2 error (things that aren't translated as well as things that shouldn't have been translated) we are capturing the vast majority of the simple problems.</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Internationalization&diff=2961 2011-12-13T10:16:29Z

<p>Masaru20100: First attempt to add content and clear things a bit</p> <hr /> <div>=Introduction=<br /> <br /> PCGen for 6.0+ releases will be attempting Internationalization. (probably not in 6.0) To deal with this we have created a Yahoo group called [http://groups.yahoo.com/group/pcgen_international pcgen_international], that is dedicated to the development of non-English language content for PCGen.<br /> <br /> Requests for translations, problems with translations, and drafts of translated data sets or other translated content, as well as anything coming up with the Internationalization of PCGen are to be discussed in that group.<br /> <br /> Anybody interested to help out with translations to other languages, but also those who are just interested in the discussion of the matter, please join the group. Those interested in joining our efforts on the Internationalization of PCGen, please make yourself known on that list.<br /> <br /> This page is an attempt to regroup the information discussed on the list.<br /> <br /> =Goal=<br /> <br /> The goal is to allow a user that doesn’t speak English to use the program. There is many things to consider to attain this goal.<br /> <br /> First the installation process should be detailed in many languages. The program UI should also be translated, and so is the data. The skills should be sorted in the UI and in the output sheet according to the natural sorting of the language. The output sheets should be in the user’s language too. The documentation need to be also in the user’s language.<br /> <br /> (bug report language and translation)<br /> <br /> Saved characters are usually reusable with a new version of the program, and the il8n should not change that. (technical: the saved elements should stay the same, should probably be keys of some sort — I have no idea what the pcg files are)<br /> <br /> imperial and meter thing (already handled in 12/2011 version, at least there is an item in the options)<br /> <br /> = Tools for il8n =<br /> <br /> To facilitate the internationalization and localization, some tools would be welcome. This tools are not necessary for the user, but are a great addition to maintainers.<br /> <br /> * a tool test to see if some entries are not translated (maybe a page that would provide statistics) it should be ui and data separated I think<br /> * keeping glossary for each languages could help (some terms are found at many places)<br /> * a tool to eliminate (or point at) line that are not used any more in the main bundle file (it is quite big and there seems to bit not displayed text)<br /> <br /> =Choices yet to be made=<br /> <br /> == Application Packaging ==<br /> <br /> As of December 2011, the application and translation of some part of the UI are bundled together.<br /> <br /> The question is should the application be bundled this way, or should there be language specific bundles.<br /> <br /> Pro of a all-in-one<br /> * user can change language on the fly<br /> * no need for explanations on how to install a new language set<br /> <br /> Cons<br /> * file could get quite big<br /> <br /> == Genre class names ==<br /> <br /> In some languages, the class names can have a genre, like the feminine of priest is priestess in English (even if Cleric is the name of the class, it’s just to illustrate the point). Do we want to use those names in the class names or do we stick to whatever name the class is?<br /> <br /> = The goals in more details for PCGen =<br /> <br /> == Default language after clean install ==<br /> <br /> The language should be the system one. To the user, if doesn’t understand the default language, it will be a pain for him to find the option to change the language.<br /> It may be even better if the user is prompted with a choice at the start of the program (or at install) in order to simplify this process. It might be displayed only if the system language doesn’t exist in PCGen.<br /> <br /> After clean install (no preference files):<br /> # PCGen determine the system language<br /> # if translation exist (and are believed good enough) switch to that language, if not, display an element to ask the user for the language to use.<br /> <br /> == Published language priority ==<br /> <br /> * The data that is not translated in a language, for example books not yet translated, should stay in the language it first get out.<br /> <br /> Books are almost never instantly translated by companies and some aren’t even translated. PCGen is also short on staff. All that means that the translations will probably be coming far after the original one.<br /> To an user that only use translated material, it will not be a problem.<br /> To an user that mix translated material, and not translated one, it shouldn’t become a problem.<br /> <br /> == Using different data languages ==<br /> <br /> * The data should be the same that the English one to avoid having to duplicate data correction.<br /> <br /> Having data in several languages could be a way to do il8n<br /> <br /> Cons<br /> * might need language specific bundles<br /> * a character used with one language cannot be seen by someone with a different language set<br /> * multiple corrections needed when correcting a bug (and if automatic generation of the translation is done, why not do it on the fly)<br /> <br /> == Separating data and UI translations ==<br /> <br /> * The interface language and the data language should be separate to allow people with book in a different language than their own to use the program more simply.<br /> To the user that speak only one language, it is no advantage. <br /> For a user which use book that are not in the language he is most comfortable with, he would put the UI and the data to different languages. Same for someone who doesn’t use translation of the data but prefer to have a program in a language he is more comfortable with.<br /> <br /> == Custom content and localization ==<br /> <br /> * It should be possible to use another language in a data to allow non English speaker to develop custom content in another language. That seem to be a problem when combined with separate language from data. A way to avoid it is to provide easy reference to create custom content.<br /> <br /> This might be a hard to make real combined with other points.<br /> <br /> =Technical Aspects=<br /> <br /> == Language choice UI ==<br /> <br /> There is already options to change the language and the distance metric used.<br /> <br /> It might be best to have a drop down list rather than radio choices.<br /> <br /> == Formatting Issue ==<br /> <br /> In Japanese, the classic number system is almost always used especially in RPG. In fact, translating numbers is not needed, what is needed is formatting numbers.<br /> <br /> That means that 10000 gets formatted 10,000 in English, 10 000 in French. In Japanese, it would either be 10,000 or 1万.<br /> <br /> In Java, this is usually done by using the NumberFormat class.<br /> <br /> == Sorting Issue ==<br /> <br /> Sorting is usually language dependent, meaning that the list of skills for example is not sorted the same way in different languages, even if the name are the same. The differences are usually minor between roman letter using languages.<br /> <br /> Actually, the international guys brought up the sorting thing again - another project would be to look into the GUI/core and replacing all the sorting of items with internationalized methods of sorting (Collators)<br /> <br /> * Use of collator to sort lists.<br /> <br /> == Unique Identifier ==<br /> <br /> Apparently there is already unique identifiers in use for items, so the idea would be to have a file with that id as the key and the translation as the value.<br /> <br /> == Tom’s technical proposal ==<br /> <br /> So looking back at the earlier thread, I never actually came back with my suggestion, so here is my perspective on localization and how it should be done. This has been made easier by some changes we have in the 6.x line, and having new UI (with what I believe to be a lot more isolation of code that does display of items) is a huge boost here to making this practical.<br /> <br /> First a few base facts as background:<br /> # For [almost] every item (Class, Skill, etc.) there is a unique identifier (generally referred to as a Key - note that if the &quot;KEY&quot; token is not used, then the Key is the name (first entry on the line in the data) (There is an exception to this we'll call problem #1)<br /> # There are basically 4 types of things that need translation:<br /> ## (2a) item names<br /> ## (2b) constants (like spell schools)<br /> ## (2c) variables (like meters/feet/etc.)<br /> ## (2d) Strings (like descriptions)<br /> ## [if anyone can think of more, let me know]<br /> # Most (but not all) tokens are &quot;unique&quot; or otherwise &quot;addressable&quot; in that they can only occur once per object. (Anyone notice that the tokens have started to be specific in the test code &amp; docs about whether they overwrite, add, etc. - this is one reason why - and yes, I've been slowly trying to make progress on this even in the 2007-2009 work) [The exception to this is problem #2)<br /> <br /> A few principles about l10n:<br /> # We must not make producing a data set materially more complicated than it is today (no requirement to put %L10NNAME% type gunk into data)<br /> # We should target an ability to tell us if l10n is complete for any given data set<br /> <br /> With that:<br /> * Following from #1, almost everything we have in the data today is &quot;addressable&quot;. By that, I mean that the OUTPUTNAME for a Skill called &quot;FooBar&quot; can be uniquely called out. The name hierarchy is something like: SKILL//Foo//OUTPUTNAME (exceptions are problems #1,2 to be addressed later)<br /> <br /> Given that, we can actually set principle #1 to &quot;The data remains unchanged&quot; (again, except for problem #2). The entire data set is produced (assume US English for a moment). We then have a unique file for l10n that has things like:<br /> * SKILL:FooBar|Oobarf-ay<br /> * SKILL:FooBar:OUTPUTNAME|ooBarOutF-ay<br /> * etc.<br /> <br /> This (to the first order) covers 2a, 2d.<br /> <br /> For 2b items, we simply have to expand the list of items (SKILL, SPELL) that we are familiar with, so we get things like:<br /> * SPELLSCHOOL:Divination|Ivinationd-ay<br /> <br /> 2c is a bit more complicated, but I can't believe it's all that bad given it's a thing many applications already do (and it's a known thing)<br /> <br /> Each file could be named in such a way that it identifies it's l10n, e.g.:<br /> * srd_de_ch.l10n<br /> * ...and probably placed in a L10N subfolder of the initial dataset.<br /> * (Note I'd recommend we be clear on where these go, AND ALSO require that NO PCC FILES (I'd recommend we say no PCC or LST, but the formal limit would be no PCC) are recognized if they are in the L10N folder... so that the initial directory parse [which is probably one of the slower parts of our boot process] can immediately ignore the l10n directory and not have to look through the file list looking for .PCC files... alternately, we have a l10n folder that is parallel to the data folder, but that then adds complication in needing new preferences to point at multiple l10n folders and requires more complicated structure within that l10n structure to identify WHICH files in that folder map to which datasets... because we can't simply say a given English word will always translate the same way - English is way too overloaded for that.)<br /> <br /> I would recommend we keep things in a small subset of files, and not try to do 1:1 for each data file (that would produce a lot of file sprawl) - but that's not my call.<br /> <br /> Addressing principle #2:<br /> * Since we have a set of items we know would need to be covered (names, certain tokens), we should be able to load those into memory, and load the l10n file into memory and compare. This should be able to produce warnings of 2 kinds:<br /> * (W1) Errors where the base data set contains things that are not translated<br /> * (W2) Errors where the translation file attempts to translate things not in the base data set<br /> * I can't imagine that utility is all that hard to write (just would need to make the list of todos)<br /> <br /> This brings us to problems #1 and #2:<br /> <br /> Problem #1: Non-unique names<br /> * We glossed over this in 6.x, but the truth is that Spell names are not unique. Some of the *RDs have duplicate names (not all, but I forget which). Same is true for languages.<br /> * (1a) Spells can theoretically be differentiated by evaluating the &quot;TYPE&quot; token for Divine, Arcane, or Psionic (those are &quot;magical&quot; items in our code.<br /> * (1b) Languages can theoretically be differentiated between &quot;Spoken&quot; and &quot;Written&quot; as those are &quot;magical&quot; types.<br /> I believe both of those forms of magic are things on our backlog of FREQs to clean up... and the reason they are on the cleanup list really for L10N (as much as it is to just cleanup the overuse of TYPE)<br /> <br /> Problem #2: Non-unique tokens<br /> * There are only a few tokens that are not unique. DESC is one of them, if I recall correctly. The probably solution here is to simply give an identifier to each token. This might require a change to LST Something like:<br /> * DESC*Overall:x<br /> * DESC*Second:x<br /> * (Note: I'm not sure this syntax works or is by any means &quot;good&quot;. DESC:OVERALL|x might be better as it avoids potential issues with using * as a reserved character - take this is a principle of what would have to happen to DESC token, not as a full-blown proposal)<br /> <br /> Note that this naming of each DESC item (And other reusable items), while it breaks the &quot;can't change the data&quot; principle actually helps as much as it hurts... it would give us the ability to do things like:<br /> * DESC:.CLEARID.Overall<br /> * or things similar to that, which is actually a neat benefit for the small overhead of pain it puts into datasets. (Which, by the way, could be converted to whatever we decide on anyway with our nifty converter, so this doesn't seem all that bad)<br /> <br /> So in my mind, the question really is: Is going slightly more than half way good enough? There are areas where we could do translation, and some areas where we need some pretty material core code changes to support it.<br /> <br /> == Translation files ==<br /> <br /> Hi Tom,<br /> <br /> Welcome BACK!!!<br /> <br /> I don't mind the solutions put forth. Unique Identifiers for DESC is no more absurd then the Unique<br /> Identifiers we'll be needing to use more than one CHOOSER on a single line. Plus, the ability to<br /> clear off a Section of DESC would be awesome.<br /> <br /> I'd vote for as complete a job as possible.<br /> <br /> Here is where I'm not understanding - where are we going to put the translation stuff?<br /> <br /> (choices I view<br /> One translation file (can get really big)<br /> One translation file per file (apparently Tom advise against this )<br /> regroup several translation in a file<br /> )</div>

Masaru20100 http://159.203.101.162/w/index.php?title=Talk:Internationalization&diff=2960 2011-12-13T08:54:32Z

<p>Masaru20100: Created page with &quot;I’m working on getting this page cleared up a bit, it’s still a work in progress. ~~~&quot;</p> <hr /> <div>I’m working on getting this page cleared up a bit, it’s still a work in progress. [[User:Masaru20100|Masaru20100]]</div>

Masaru20100