[Bf-docboard] Translations Tracker

[Campbell Barton]

Tools like this are handy to keep in our repos.

Made some changes:
- Show percentages from 0-100, 100 when complete.
- Use argparse for argument handling.
- Add a --quiet option to only report the summary.
- Avoid dictionary access for typical vars.
- Use a set for listing fuzzy files (not to print then many times)
- Move strings for reporting inline.
- Can take multiple path arguments (report for each language).
- Take report as an argument,
  (to override print if we want to use multi-processing later).

Gives overall 6x speedup here,

Committed https://developer.blender.org/rBM1029

On Thu, Dec 24, 2015 at 10:39 AM, Aaron Carlisle
<carlisle.aaron00 at gmail.com> wrote:
> We may want to include this in trunk like we do with other .py tools in the
> main documentation repository, now that it is in python and everyone is able
> to use it.
>
> On Wed, Dec 23, 2015 at 1:11 PM, Anton Felix Lorenzen <anfelor at web.de>
> wrote:
>>
>> Hi all,
>>
>> I re-wrote the script in python,
>> it's now around 360 times as fast using python3
>> (and even 675 times when using pypy).
>>
>> I overused grep before,
>> so it took the script 9 minutes to finish,
>> now it runs in approx. 1 second.
>>
>> It also shows files with fuzzy strings in it
>> and should also run on windows.
>>
>> Sincerely,
>> Anton Felix Lorenzen
>>
>> Usage:
>> |- Paste into file named translations.py
>> |- "cd" into folder
>> |- execute python translations.py "C:\path\to\LC_MESSAGES"
>>
>> Script:
>>
>> #!/usr/bin/env python2
>> # -*- coding: utf-8 -*-
>> #
>> #  Translations Tracker 1.0
>> #  works best with pypy
>> #
>> #  Copyright 2015 Anton Felix Lorenzen <anfelor at web.de>
>> #
>> #  Permission is hereby granted, free of charge,
>> #  to any person obtaining a copy of this software
>> #  and associated documentation files (the "Software"),
>> #  to deal in the Software without restriction,
>> #  including without limitation the rights to use, copy,
>> #  modify, merge, publish, distribute, sublicense,
>> #  and/or sell copies of the Software,
>> #  and to permit persons to whom the Software is furnished to do so,
>> #  subject to the following conditions:
>> #
>> #  The above copyright notice and this permission notice
>> #  shall be included in all copies or substantial portions of the
>> Software.
>> #
>> #  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
>> #  EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
>> MERCHANTABILITY,
>> #  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
>> #  IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
>> CLAIM,
>> #  DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
>> #  TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
>> SOFTWARE
>> #  OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
>>
>> import os
>> import codecs
>> import sys
>>
>> if(sys.argv and sys.argv[1]):
>>         path = sys.argv[1]
>> else:
>>         path = "~/locale/fr/LC_MESSAGES"
>>
>> # Configuration
>> fileinfo  = '{0[file]:>3} empty, {1[file]:>3} fuzzy of {2[file]:>3}; or
>> {3:.3f}% in {4}'
>> allinfo   = 'Summary: {0[all]} empty of {1[all]}; or {2:.3f}%'
>> fuzzyinfo = 'Fuzzy:   {0[all]} fuzzy strings in {0[filelist]}'
>>
>> msgstrs = {'file': 0, 'all':0}
>> empty_msgstrs = {'file': 0, 'all': 0}
>> fuzzy = {'file': 0, 'all': 0, 'filelist': []}
>> lastLineWasEmptyMsgstr = False
>>
>> def forAllInMap(m, fnk, cond):
>>         for key in m:
>>                 if(cond(m[key])):
>>                         m[key] = fnk(m[key])
>>
>> def forAllIntsInMap(m, fnk):
>>         forAllInMap(m, fnk, lambda x: type(x)==int)
>>
>> def increaseAllInMap(m):
>>         forAllIntsInMap(m, lambda x: x+1)
>>
>> def decreseAllInMap(m):
>>         forAllIntsInMap(m, lambda x: x-1)
>>
>> for (root, subs, files) in os.walk(path):
>>         for name in files:
>>                 msgstrs['file'] = -1 # First lines contain a "fake" msgstr
>>                 msgstrs['all'] -= 1
>>                 empty_msgstrs['file'] = 0
>>                 fuzzy['file'] = 0
>>                 if name.endswith('.po'):
>>                         for line in codecs.open(os.path.join(root, name),
>> encoding='utf8'):
>>                                 if(line[0] == 'm'):
>>                                         if(line[0:6] == 'msgstr'):
>>                                                 increaseAllInMap(msgstrs)
>>                                                 if(line[0:9] == 'msgstr
>> ""'):
>>
>> increaseAllInMap(empty_msgstrs)
>>
>> lastLineWasEmptyMsgstr = True
>>                                 else:
>>                                         if(line[0] == '"'):
>>
>> if(lastLineWasEmptyMsgstr):
>>
>> decreseAllInMap(empty_msgstrs)
>>
>> lastLineWasEmptyMsgstr = False
>>                                         else:
>>                                                 lastLineWasEmptyMsgstr =
>> False
>>                                                 if('fuzzy' in line):
>>
>> increaseAllInMap(fuzzy)
>>                                                         fuzzy['filelist']
>> += [name]
>>
>>                 print(fileinfo.format(empty_msgstrs, fuzzy, msgstrs,
>>                         float(empty_msgstrs['file'] + fuzzy['file']) /
>> msgstrs['file'],
>>                         os.path.join(root[len(path):], name)))
>>
>> print(allinfo.format(empty_msgstrs, msgstrs, float(empty_msgstrs['all'])
>> / msgstrs['all']))
>> print(fuzzyinfo.format(fuzzy))
>> _______________________________________________
>> Bf-docboard mailing list
>> Bf-docboard at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-docboard
>
>
>
> _______________________________________________
> Bf-docboard mailing list
> Bf-docboard at blender.org
> http://lists.blender.org/mailman/listinfo/bf-docboard
>



-- 
- Campbell