python
#define python: \
I------------------------------------------------------------------------------------\
I------------------------------------------------------------------------------------\
I------------------------------------------------------------------------------------\
I /$$$$$$$ /$$ /$$ \
I | $$__ $$ | $$ | $$ \
I | $$ \ $$ /$$ /$$ /$$$$$$ | $$$$$$$ /$$$$$$ /$$$$$$$ \
I | $$$$$$$/| $$ | $$|_ $$_/ | $$__ $$ /$$__ $$| $$__ $$ \
I | $$____/ | $$ | $$ | $$ | $$ \ $$| $$ \ $$| $$ \ $$ \
I | $$ | $$ | $$ | $$ /$$| $$ | $$| $$ | $$| $$ | $$ \
I | $$ | $$$$$$$ | $$$$/| $$ | $$| $$$$$$/| $$ | $$ \
I |__/ \____ $$ \___/ |__/ |__/ \______/ |__/ |__/ \
I /$$ | $$ ((((python))) \
I | $$$$$$/ \
I \______/ \
I------------------------------------------------------------------------------------\
I------------------------------------------------------------------------------------\
I------------------------------------------------------------------------------------I
$ python -c 'import this'
"There should be one-- and preferably only one --obvious way to do it."
— The Zen of Python
TODO:
as; async; await; finally; is; nonlocal; yield;
• most of this documentation assumes python3 or higher
• forget semicolons;
• whitespace sensitive
• interpreted language
• the python3 interactive shell uses ">>>" as ${PS1} and "..." as ${PS2} by default; except to see it in examples
if you very creatively name your module testing files [module].py that WILL break shit as it will try to import itself;
"<-->" signals indentation, cause i repeat, this language is whitespace sensitive
python
#define python-programs:: \
I#######################################\
I ___ \
I | _ \_ _ ___ __ _ _ _ __ _ _ __ ___ \
I | _/ '_/ _ \/ _` | '_/ _` | ' \(_-< \
I |_| |_| \___/\__, |_| \__,_|_|_|_/__/ \
I |___/ \
I#######################################I
************
pip3:
************
A tool for installing and managing Python packages.
do not run pip as root; such action can break system tools, leading you to speedrun the Linux installation (again)
pip3 [operation]
install : installs packages
[package] : [package]
— r [file] : packages listed in [file]
uninstall
search [package] : searches for [package]
list : lists installed [packages]
**********
***************
python3:
***************
python3 ([options]) [file]
• not providing any arguments will result in the python interpreter being run interactively
• [file] being a python script
— c '''<string>''' : pass program as <string>
— E : ignore python environment variables
*************
*****************
ipython3:
*****************
• python3 wrapper; fully compatible
• slow as fuck
— adds the following features:
• syntax highlighting
• auto history
• dynamic object introspection
• auto completion
• system shell access
***************
************
pudb:
************
> its broken; step does not step.
>interactive python debugger
>tui
>looks great
>minimal typing redundancy friendly
**********
Other_notable_implementations:
• IronPython by Microsoft
• Anaconda
FILES:
.py : Python script
.pyc : Bytecode compiled from a Python script; usually created by the interpreter automatically and stored inside a __pycache__
.ipynb : "Interactive PYthon Note Book"; JSON export of a Jupiter-lab session
__pycache__/ : a folder create by the interpreter at runtime specifically for caching
requirements.txt : new line separated list file pip packages used for storing an environment
python
#define python-language:: \
I##########################################\
I _ \
I | | __ _ _ _ __ _ _ _ __ _ __ _ ___ \
I | |__/ _` | ' \/ _` | || / _` / _` / -_| \
I |____\__,_|_||_\__, |\_,_\__,_\__, \___| \
I |___/ |___/ \
I##########################################I
FILES: FILES:
Extentions:
py : python script
pyc : compiled python code
Runtime:
.python_history : interactive python interpreter history file
Commenting: Commenting:
# [stuff] : single line
'''[stuff]''' : multi line; more of a bug then a feature tho
Importig: Importig:
• including really
• module-s import-ed in import-ed module-s are not visible in the base file
{ import [lib]/[python file]
[imported python file (no extension)].[function from imported python file]()
}
or
{ from [python file (no extension)] import [function from imported file]
[imported file]()
}
or
{ from [python file (no extension)] import *
[function]()
}
Operators: Operators:
• operator-s complying with C/C++ do not have a description, see that AT /C++/Operators
Logical:
and
or
not
•
<
==
!=
• =
<=
— in
[a] in [b] : whether [a] equals to any member of [b]
[a] not in [b] : not ([a] in [b])
— is
[a] is [b] : are [a] and [b] the same object;
being equal in value isn't enough
[a] not is [b] : not ([a] is [b])
{ >>> y = ["python", "C++", "C"]
• >> x = y
• >> x is y
True
• >> x = ["python", "C++", "C"]
• >> x is y
False
}
Artimetric:
+
+=
. -
. -=
%
%=
*
*=
** : power
**= : power assignment
/ : divide to float
/= :
; divide to int
;
Bitwise:
&
|
^
. >>
<<
Misc:
*[list || tuple] : unpacking operator; translates [list || tuple] to a literal of its elements
{ >>> def repPrint(times, _str):
... for i in range(times):
... print(_str)
...
• >> myTuple = {3, "This feature is legit cool as fuck."}
• >> repPrint(*myTuple)
This feature is legit coll as fuck.
This feature is legit coll as fuck.
This feature is legit coll as fuck.
}
**[dictionary] : dictionary unpacking operator
Logic: Logic:
— if
{ if [bool]:
<-->[...]
([elif])
([else])
}
○ [elif]
• else if
{ elif [bool]:
<-->[...]
}
○ [else]
{ else:
<-->[...]
}
• "True" != "true"; (NameError: name 'true' is not defined)
Loops: Loops:
— for
• [variable] will take up the values of [list] or [range(<int>)] as it goes throught the elements
• technically its a foreach loop
{ for [variable] in [list]:
<-->[commands]
<-->[...]
}
— while
{ while [logic]:
<-->[commands]
}
• break && continue works as if a sane person had written this shit
Literals:
Int:
[digit|_]*
{ 1_000_000 }
Float:
<int>.<int>
<int>.
.<int>
{ 3.14; .1; 2. }
String:
'[...]' : string literal; '"'s dont have to be escaped
"[...]" : string literal; '''s dont have to be escaped
r<string_literal> : raw string literal; escape sequences are not processed; most commonly used when writing regex
f<string_literal> : format string; like string.format where the current scope is passed in
Variables: Variables:
• dynamically typed
• variable type can change
• one cant declare a variable before assigning a value to it, therefor { myString = "" } and { myList = [] } are common sights
• variable names must start with a letter or the underscore char
○ types
○ basic
• bool; const
• int; const
• long; const
• float; const
• complex (scientific numbers){ 3e+26 }; const
— iterable:
• a container type which can be iterated over
• from now on referred to as "itr"
○
• string; const
— unicode by default (used to be ascii prior to python3)
— list (like if a c++ vector would allow different variable types)
• mutable
• [list][num] : [num]-th member of [list]
• [list][num:] : all members of [list] from [num]; including [num]
• [list][:num] : all members of [list] till [num]; not including [num]
• [list][num1:num2] : members between [num1] and [num2]; including [num1], but not including [num2]
• negative [num]-s start from the end
• you can go out of range tho; (nor consistency nor wholeness)
{ [name] = [] }
— tuple (a const list)
• mutable
{ [name] = () }
— dictionary
• mutable
{ [name] = {<key> : <value>} }
— casting:
[type_func]([variable])
• does what you would expect (on heroin)
• [type_func] is a builtin function see AT "../Builtin functions"
{ int("9")
str(3.14)
}
Scoping:
• variables declared at the script level are global
• variables declared inside functions are local to the function
• variables declared inside a class are local to instances
• global variables are always readable
global <variable> : specifies that the symbol in the current scope is global;
can be used to write the value of a global variable from
inside a function;
the symbol referenced may or may not be already defined,
assuming its not, attempting to read its value will throw
Comprehensions: Comprehensions:
• can return either a list, dictionary or set
[ <select> <from> (<where>) ] : list comprehension
{ <select> <from> (<where>) } : set comprehension
{ <select-dict> <from> (<where>) } : dictionary comprehension
execution:
1. Get next item (break if none) <--+
2. Filter <--|-+
3. Append eval-d return value <--|-|-+
4. goto 1. | | |
+---------------------+ | |
| | |
| +----------------+ |
| | |
+-------|------|------------------+
| | |
| | |
| | |
| | |
V V V
[ <select> <from> <where> ]
<from>:
for <...> in <...>
• how it works on any iterable {files}
<where>
if <bool>
• optional
• filters whether an element should be added or not
<select>
<statement>
• any statement
• the return value is what gets appended to the generated list
<select-dict>
<statement> : <statement>
• any 2 statements separated by a colon
• the statements return values become key-value pairs
{ # Unrealistic example showing way too many things at once
>>> [int(x) + 1 for x in "20232805080" if x != '0']
[3, 3, 4, 3, 9, 6, 9]
}
Functions: Functions:
{ def [name](([parameters])):
<-->[command]
<-->[...]
<-->(return [variable])
}
• default values work C style
• call it as normal ( [name](([parameters])) ) { print("Heyo") }
— nested functions are allowed
• nested functions can implicitly reference the parents variables
lambda:
lambda [parameters] : <expression>
• creates an anonymous function object
• can contain only one expression
• the expressions value is returned
• mostly syntactic sugar for quick callbacks
>>> list(filter(lambda x : x >= 5, [1, 5, 3, 8, 9, 4, 2, 6]))
[5, 8, 9, 6]
Type_members: Type_members:
<string>:
.upper() : returns <string> with CAPS
.lower() : returns <string> with all lower case letters
.title() : returns <string> with CAPS first letter of words
.split([string1], <int>) : returns list with <string> split at [string1] (if none is specified at spaces)
<int> specifies the maximum list elements to split into (indefinite if none is specified)
.join()
.find(<string>) : returns position of <string>s first occurrence or -1
.rfind(<string>) : returns position of <string>s last occurrence or -1
.replace([string-1], [string-2]) : returns this with all occurrences of [string-1] replaced with [string-2]
.isalnum() : returns whether <string> consists only from alpha numeric chars
.isalnum() : returns whether <string> consists only from whitespace chars
.format([name] = [val]) : returns <string> with placeholder [name] switch-ed to [val];
placeholders follow the format: "{name}"
string[[start]: [end]: [step]] : returns a substring
string[start:end] : get all characters from index start to end-1
string[:end] : get all characters from the beginning of the string to end-1
string[start:] : get all characters from index start to the end of the string
[start] : The starting index of the substring. The character at this index is included in the substring.
If start is not included, it is assumed to equal to 0.
[end] : The terminating index of the substring. The character at this index is NOT included in the substring.
If end is not included, or if the specified value exceeds the string length,
it is assumed to be equal to the length of the string by default.
[step] : Every ‘step’ character after the current character to be included. The default value is 1.
If the step value is omitted, it is assumed to equal to 1.
[list]:
.append([value]) : hozzáad a végéhez
.pop([num]) -> [element] : removes [num]-th member; if left blank, removes last
[dictionary]:
.keys()
.values()
[file]:
.write([typename presumed by mode]) : writes [typename presumed by mode] to [file]
.read() : returns string
.readline() : returns line of string
.close() : duh; should always be called
Builtin_functions: Builtin_functions:
• the most important functions are marked with a 'Ж' at the begining of their line
__import__([module], : import-s [module] with [dictionary1] globals and [dictionary2] locals;
[dictionary1] = None, [dictionary2] = None, this function is invoked by the import keyword under the hood
[list] = (), <int> = 0)
abs(<int>) : returns absolute value of <int> or member __abs__()
all([itr]) : returns True if none of [itr]s elements are False
any([itr]) : returns True if any of [itr]s elements are True
ascii(<string>) : returns a ascii valid, unicode escaped representation of <string>
bin(<int>) : returns <int> or member __index__() converted to python valid binary format (prefixed with "0b")
Ж bool([var]) : returns [var] converted to bool
breakpoint() : import-s pdb (see AT "../STD modules/pdb") and calls set_trace(); ie. starts a debugger
bytearray([],[],[])
bytes([var]) : returns a immutable bytes object initialized with the given size and data.
callable([object]) : returns whether it is possible to call [object] (no success guaranteed)
Ж chr(<int>) : returns char represented with value <int>
classmethod()
compile()
complex()
delattr([object], <string>) : delete-s [object]s <string> member; { delattr(x, "foobar") == del x.foobar }
Ж dir([])
divmod([], [])
enumerate([list]) : creates a list of tuples with 0 to N and [list]-s elements;
primarily used in for loops while an accumulator is also needed
lazily evaluated;
>>> list(enumerate(["a", "b", "c", "d", "e"]))
[(0, 'a'), (1, 'b'), (2, 'c'), (3, 'd'), (4, 'e')]
eval([], [], [])
exec([], [], [])
filter([func], [itr]) :
Ж float([var]) : returns [var] converted to float
format()
frozenset([itr])
getattr([object], <string>) : returns [object] member called <string> { getattr(x, "foobar") == x.foobar };
fuck getters and setters in C++, but this is actually a nice feature and good practice
(see also AT "./getattr")
globals() : returns a directory of the current global symbol table
hasattr([object], <string>) : returns whether [object] has a member called <string>
help([object]) : prints info about class
hex(<int>) : returns <int> or member __index__() converted to python valid hexidecimal format (prefixed with "0x")
id([object]) : returns a unique int corresponding [object]; objects with non-overlapping lifetimes
may have the same id() value
Ж input((<string>)) : reads stdin for a line if no argument is given, else outputs <string> to stdin
Ж int([var], <int> = 10) : returns [var] converted to int in base <int>
isinstance([object], [class]) : returns whether [object] is an instance of [class]
issubclass([class1], [class2])
iter([object], [])
Ж len(<string>) : length of <string>
locals() : returns a directory of the current local symbol table
Ж list([itr]) : lists(?) [list]; (instead of treating it as a memory address)
Ж map([func], [itr])
max([itr1](, [itr2])) : returns the largest element from [itr](s)
min([itr1](, [itr2])) : returns the smallest element from [itr](s)
next()
oct(<int>) : returns <int> or member __index__() converted to python valid octal format (prefixed with "0o")
open([file name], "[mode]") : returns file object
○ mode
x : create (fail if the file already exists)
r : read
w : write
a : append
t : text mode; default
b : binary mode
+ : updating (r + w)
ord([char] || <int>) : return either unicode int representation of [char] or unicode char representation of <int>
pow([int1], [int2](, [int3])) : returns [int1] as base raised to exponent [int2]; [int3] is a modulo,
pow(i, h, g) is more efficent than ( pow(i, h) % g )
print([var]) : prints [var] to stdout
property()
Ж range(<int>) :
Ж range([int1], [int2]) :
repr([object]) : returns a string representation of [object]; ideally this string can be fed to eval()
to create an equivalent object, other wise string is enclosed as: '<'[...]'>'
reversed([])
round([num1](, [num2])) : returns [num1] rounded to the nearest int or to [num2] decimal places
Ж set([itr]) : returns a set constructed from [itr]
setattr([object], <string>, [var]) : sets <string> member of [object] to [var] (see also AT "./getattr")
sorted()
staticmethod()
sum()
super()
Ж tuple([itr]) : returns a tuple constructed from [itr]
type([var]) : returns [var]s type
vars([])
get([list || tuple]) : sorts and removes duplicates
zip([list_a], [list_b]) : returns a list of lists, which's members are paired together
• if cant pair, will purge
• can take unlimited number of lists
{ # Zip Used
letters = ["a", "b", "c", "d", "e"]
numbers = ["1", "2", "3", "4", "5"]
print(zip(letters, numbers))
# Output
[('a', '1'), ('b', '2'), ('c', '3'), ('d', '4'), ('e', '5')]
}
Classes: Classes:
• standardly defined, implements (multiple) inheritance and polymorphism
{ >>> class [name]:
... <-->[...]
}
○ methods( ie. member functions)
• the object is explicit-ly passed -> the first argument is always the object itself
• the convential name for the first argument is "self"
{ // this example presupposes an MP3 class (m) which has a "len" member
>>> class music:
... <-->m = readMP3("Seether - 03. - Karma and Effect (2005).mp3")
... <-->def length(self):
... <--><-->return self.m.len
}
— __init__:
• the constructor really
{ >>> class a:
... <-->def __init__(self, b):
... <--><-->print(b)
...
>>> n = a("Hello world")
Hello world
}
Exceptions: Exceptions:
• every interpreter error will raise an exception and therefor can be caught
{ try:
[...]
except [exception class] as [name]:
[...]
}
{ raise [...]
}
class BaseException:
• all exceptions must be derived from it
• can be converted to string
Modules:
— a module is an external script used by the main one with the following properties:
• no hardcoded path is provided
• every symbol is optionally accessable
• every symbol is namespace-d by default
• the module script is aware its being used as a module
• a python module is just a script
• modules must end with the ".py" extension
• the python script at the root of the execution has the variable __name__ := "__main__",
unless this is true, we can persume our script is runnning as a module
Packages:
• logical module wrapper
• does not contain data by itself
• accomplished by creating a folder on the filesystem level
Import:
import <module> (as <alias>) : import <module> namespace-d;
optionally rename the namespace as <alias>
from <module> import * : import every public symbol from <module>
into the global namespace
from <module> import <symbol>(, <symbol-2>)* : import the listed symbols from <module>
into the global namespace
• <module> must not contain the extension ".py"
. ### Intuitiv python module tl;dr ###
{ $ tree
.
├── main.py
├── mymodule.py
└── mypackage
└── mypackagedmodule.py
2 directories, 3 files
$ for i in **/*.py; do
bat $i
done
───────┬────────────────────────────────────────────
│ File: main.py
───────┼────────────────────────────────────────────
1 │ import mymodule
2 │ import mypackage.mypackagedmodule as mpm
3 │
4 │ mymodule.f()
5 │ mpm.f()
───────┴────────────────────────────────────────────
───────┬────────────────────────────────────────────
│ File: mymodule.py
───────┼────────────────────────────────────────────
1 │ def f():
2 │ return 0
3 │
4 │ if __name__ == '__main__':
5 │ print("heyo")
6 │ else:
7 │ print("I am a module.")
───────┴────────────────────────────────────────────
───────┬────────────────────────────────────────────
│ File: mypackage/mypackagedmodule.py
───────┼────────────────────────────────────────────
1 │ def f():
2 │ return 1
───────┴────────────────────────────────────────────
$ python main.py
I am a module.
$ python mymodule.py
heyo
}
#
Escape_sequences:
• all python escape sequences are C/C++ complient,
therefor see description AT "/C++/Escape\ Sequences"
\a
\b
\f
\n
\r
\t
\v
\" "
\'
\\
\[d][d][d]
\x[d][d]
— recognized only inside string literals:
\u[d][d][d][d]
\U[d][d][d][d][d][d][d][d]
\N[name] : unicode char named [name]
### Guide to Python Venvs ###
• "Virtual ENVironment"
• a venv is a self contained python installation
• they help to avoid system pollution
• guarantee a state which has no dependency conflicts
• there is a builtin "venv" module
1. Creation
• the entirety of a python venv resides in an arbitrarily named folder
$ python -m venv <venv_name>
2. Activation/Deactivation
• "activating" a venv means to manipulate a shell's behaviour
to operate with the specific python installation which the venv provides
• activating works by redefining ${PATH}, defining functions and so on
+----------+------------+-----------------------------------------+
| Platform | Shell | Command to activate virtual environment |
+----------+------------+-----------------------------------------+
| | bash/zsh | $ source <venv>/bin/activate |
| POSIX | fish | $ source <venv>/bin/activate.fish |
| | csh/tcsh | $ source <venv>/bin/activate.csh |
| | PowerShell | $ <venv>/bin/Activate.ps1 |
| Windows | cmd.exe | C:\> <venv>\Scripts\activate.bat |
| | PowerShell | PS C:\> <venv>\Scripts\Activate.ps1 |
+----------+------------+-----------------------------------------+
• since activation is a shell operation,
the "session"s lifetime will never exceed the lifetime of the shell,
each shell requires activation to use the venv
• activation will change the prompt, this is a semi reliable way to tell
whether you are inside a virtual environment
{
$ source myvenv/bin/activate
(venv) $
}
• explicit deactivation is possible:
$ deactivate
Venvs_in_projects:
• you should
• makes installation by others infinitely less painful
— do NOT bloody try to move/share them:
"Warning: Because scripts installed in environments should not expect\
the environment to be activated, their shebang lines contain the absolute paths\
to their environment’s interpreters. Because of this, environments are\
inherently non-portable, in the general case."
Shipping_a_venv:
1. Set it up
$ python -m venv <my_venv>
$ source <my_venv>/bin/activate
(venv) $ python -m pip install <package>+
2. Dump it
$ (venv) python -m pip freeze > requirements.txt
3. Ship it
• add your "requirements.txt" to your version control system
• do NOT track "<my_venv>", and while you're at adding things to your .gitignore,
do the same with "__pycache__", damn it
Restoring_a_venv:
1. Recreate
$ python -m venv <my_new_venv>
2. Activate
$ source <my_new_venv>/bin/activate
3. Populate
(venv) $ python -m pip install -r requirements.txt
#
Misc_keywords: Misc_keywords:
assert [condition][, <string>] : stops program and writes <string> as error message (if any given, its optional)
(do not include the ',' if you dont give an error message)
del [object] : frees memory of object; "deletes" it
pass : a statement that does nothing; used when a statement is required, but no action is desired:
{ >>> def myFunction(): #TODO: implement
... <-->pass
}
python_std_modules
#define python_std_modules:: \
I=========================================================================\
I=========================================================================\
I _____ ___________ ___ ______________ _ _ _ _____ _____ \
I / ___|_ _| _ \ | \/ | _ | _ \ | | | | | ___/ ___| \
I \ `--. | | | | | | | . . | | | | | | | | | | | | |__ \ `--. \
I `--. \ | | | | | | | |\/| | | | | | | | | | | | | __| `--. \ \
I /\__/ / | | | |/ / | | | \ \_/ / |/ /| |_| | |____| |___/\__/ / \
I \____/ \_/ |___/ \_| |_/\___/|___/ \___/\_____/\____/\____/ \
I=========================================================================\
I=========================================================================I
~~~~~~~~~~~~~~~~
argparse: argparse:
~~~~~~~~~~~~~~~~
import argparse
Functions:
Classes:
ArgumentParser:
Constructor:
(description = <string>)
Member_funtions:
add_argument(<string>, type=<typename>, dest=<string> default=[value])
~~~~~~~~~~~~~~
ctypes: ctypes:
~~~~~~~~~~~~~~
import ctypes
• ctypes is a foreign function library for Python. it provides C compatible data types,
and allows calling functions in DLLs or shared libraries.
Data types:
-------------------------------------------------------------------------
| C_Type | Python_Type | ctypes_Type |
-------------------------------------------------------------------------
| char | 1-character string | c_char |
| wchar_t | 1-character Unicode string | c_wchar |
| char | int/long | c_byte |
| char | int/long | c_ubyte |
| short | int/long | c_short |
| unsigned short | int/long | c_ushort |
| int | int/long | C_int |
| unsigned int | int/long | c_uint |
| long | int/long | c_long |
| unsigned long | int/long | c_ulong |
| long long | int/long | c_longlong |
| unsigned long long | int/long | c_ulonglong |
| float | float | c_float |
| double | float | c_double |
| char* (NULL terminated) | string or none | c_char_p |
| wchar_t* (NULL terminated) | unicode or none | c_wchar_p |
| void* | int/long or none | c_void_p |
-------------------------------------------------------------------------
~~~~~~~~~~~
dir: dir:
~~~~~~~~~~~
dir([lib]) : returns list of [lib] members; (functions, classes, etc.)
getmembers([class]) : returns list of [class] members
callable([function]) : returns bool; value deppends on whether the fuction can be called
isinstance([object], [class]) : returns bool; value deppends on whethet [object] is a instance of [class]
issubclass([object], [class]) : as ABOVE, but bout inheritance
eval()!!!!!!!!!!
~~~~~~~~~~~~~
numpy: numpy:
~~~~~~~~~~~~~
import numpy as np
Functions:
array([list], <typename> = float) : returns an ndarray object; can be multidimentional by [list] being a list of lists
{ array([[1,2,3],[1,2,3]]) }
arrange([int]) : returns array created from range([int])
arrange([int-step], [int-limit]) : returns array created from range(0, [int-limit], [int-step])
empty([shape], <typename> = float) : returns an ndarray object according to the arguments with uninitialized values (ie. random)
full([shape], <typename>) : returns an ndarray object according to the arguments with all values initialized to value <typename>
zeros([shape]) : returns an ndarray object according to the [shape] with all values initialized to 0
fill_like([array], [value1]) : as ABOVE; will have the same dimensions as [array], but filled with [value1]
empty_like([array])
linspace([int-start], [int-end], [int-step])
Classes:
ndarray:
Member_funtions:
.shape() : returns [shape] size of each dimension
.ndim() : returns number of dimensions
.dtype() : returns typename
.itemsize() : returns size of element
.size() : returns over all element number; can be used as { a[0].size }
.copy() : return an identical function; (so
{ a = array([1,2,3])
b = a.copy
}
doesnt result in b becoming a pointer)
Operators:
operator[] : works exactly like in lists, ':' features included
[shape]:
• an int or a tuple of ints
• each int means the length of _a_ dimension
~~~~~~~~~~~~~~~~~~
matplotlib: matplotlib:
~~~~~~~~~~~~~~~~~~
import matplotlib
Matplotlib_hello_world:
{ import numpy as np
import matplotlib
import matplotlib.pyplot as plt
x_values = np.linspace(-10, 10)
y_values = x_values**2
plt.plot(x_values, y_values)
plt.show()
}
import matplotlib.pylot as plt
plot(x : [num], y : [num])
title(string)
xlabel(string)
ylabel(string)
~~~~~~~~~~
os: os:
~~~~~~~~~~
import os
Variables:
name : the name of the operating system dependent module imported ("posix", "nt", "java")
environ[<string>] : mapping object; returns corresponding value to environment variable <string>;
initialized at start up, later changes are not reflected
Functions:
system([command]) : runs system commands from shell
_exit(<int>) : exits with code <int>
Filesystem_manipulation:
mkdir([[path]name]) : creates new dir
rmdir([[path]name]) : removes dir
rename([name1], [name2]) : renames [name1] to [name2]
chmod([file], <int>) : change [file] mode to <int> see;
System_information:
ctermid() : returns the full path to the controlling terminal { /dev/tty }
getcwd() : returns path to Current Working Directory
chdir([path]) : changes working directory; (i dont think you can have multiple instances)
listdir([path]) : returns list; same as ls
stat([file]) : returns stats of file; (inode protection mode; inode number; device inode resides on; user id of the owner;
group id of the owner; size in bytes of a plain file; time of last access;
Time of last modification; creation time || metadata change)
Path:
.join([string1], [stringN]) : joins strings together, to form a path
.basename([path]) : returns file name from full path
.dirname([path]) : returns path from full path of file; (removes file from path)
.exists([path]) : returns bool; checks whether [path] exists
.isfile([path]) : returns bool; checks whether [file] exists
files:
stdout
stdin
stderr
~~~~~~~~~~~
pdb: pdb:
~~~~~~~~~~~
import pdb
• "Python DeBugger"
• gdb wrapper specifically for python
• the debugger accepts gdb commands; see AT "/?!"
Functions:
set_trace() : break-s and starts an interactive debugging session
~~~~~~~~~~~~
time: time:
~~~~~~~~~~~~
import time
functions:
sleep(<int>) : sleep (wait) <int> seconds
~~~~~~~~~~~~~~
timeit: timeit:
~~~~~~~~~~~~~~
import timeit
• used for measuring execution time of python code spinets
• all functions use their own python environment, ie. they cant access previous code by default;
if such thing is desired it must be import-ed explicitly
funtions:
timit([string-1], [string-2], : runs [string-2], then runs [string-1] <int> times using [timer] for
[timer] = time.perf_counter(), measuring time
<int> = 1000000)
{ from timeit import *
def uselessProcess():
i = 0
for h in range(1000):
i = 2*h
mycode = '''def uselessProcess():
i = 0
for h in range(1000):
i = 2*h
'''
print(timeit("uselessProcess()", "from __main__ import uselessProcess")) # importing uselessProcess explicitly
print(timeit("uselessProcess()", mycode)) # passing uselessProcess as a string
print(timeit("uselessProcess()")) # ERROR -> NameError: name 'uselessProcess' is not defined
}
~~~~~~~~~~~~
math: math:
~~~~~~~~~~~~
import math
global variables:
pi : 3.141592653589793
functions:
sqrt([num]) : square root of num
~~~~~~~~~~~~
sys: sys:
~~~~~~~~~~~~
import sys
global variables:
argv : a list; the first element is the programs name, the others are command line arguments
{ python3 sys.py -s => argv[1] == '-s' }
no "argc" use len(argv)
platform : duh
~~~~~~~~~~~~~~~
getpass: getpass:
~~~~~~~~~~~~~~~
getpass() : takes and returns input from console without echo
~~~~~~~~~~~~~~
random: random:
~~~~~~~~~~~~~~
functions:
random() : returns random number (probably float)
randuniform([min],[max]): returns random number between [min] and [max]
randint([[min], [max]]) : returns random int, between [min] and [max] if specified
shuffle([list]) : shuffles [list] by reference; returns "None"
choise([list]) : returns a random element from [list]
choices([list][, weights=[list]][, k=<int>]) : returns k <int> random elements from [list]; weights changes the
probability of the elements for being selected, you give a list full of ints to do so
~~~~~~~~~~~~~~
regex: regex:
~~~~~~~~~~~~~~
import re
Regular Expressions
Functions:
• return None on fail
compile([re](, [flags])) : returns Regex object { re.compile("\Amyword\d\s", re.A) }
search([re], <string>(, [flags])) : returns a match object from the first occurence of [re] in <string>
match([re], <string>(, [flags])) : returns a match object if [re] matches the beginning of the <string>
fullmatch([re], <string>(, [flags])) : returns a match object if [re] matches the whole of <string>
split([re], <string>(, <int>, (, [flags]))) : returns <string> split into a list at first <int> or all occurrences of [re]
findall([re], <string>(, [flags])) : returns a list of strings constructed from occurrences of [re]
finditer([re], <string>(, flags)) : returns a list of indexes where [re] was found in <string>
escape(<string>) : returns <string> with all chars interpretable as regex special chars escaped
{ re.escape("http://python.org") -> "http://\.python\.org" }
sub([re], [string1], [string2](, <int>(, [flags]))) : returns [string2] with first <int> or all occurrences of [re] replaced with [string1]
sub([re], [string1], [string2](, <int>(, [flags]))) : same as ABOVE, but returns a tuple with string and number of substitutions
purge() : clear regex cache
flags:
• controls how [re] behaves
• logical or them to use multiple { flags = re.M | re.I }
A : Ascii only
DEBUG : display Debug information about compiled expression
I : Ignore case
L : make \w, \W, \b and \B Locale dependent
M : Multiline; makes '^' and '$' match start of and end of lines
S : makes '.' match '\n's too
X : allows [re]s to be separated by spaces, which will get discarded
Regex:
Member_functions:
— functions being almost identical to the modules functions,
but using *this and without taking a [re] and [flags] in all cases
search()
match()
fullmatch()
split()
sub()
subn()
— these functions now have the added ability to take two extra int arguments
as positions (to-from) to limit the search area in <string>
findall()
finditer()
Member_variables:
flags : the [flags]
groups : ?!
groupindex : ?!
pattern : the string from which [Regex] was compiled from
re:
• can be logically operated on
• can be freely parentheses (see AT match why its useful);
inside the parentheses đP<"<string>">đ syntax can be used to name a regex
Special_characters:
. : any char except '\n'
^ : beginning of string
$ : end of string
[re]* : match any number of repetitions of [re]
[re]+ : match 1 or more repetitions of [re]
[re]? : match 0 or 1 repetitions of [re]
Special_strings:
• the optional '^' is a logical not sign in this context
[re]{<int>} : match exactly <int> repetitions of [re]
[re]{[int1], [int2]} : match from [int1] to [int2] repetitions of [re]
[(^)[chars]] : match any of [chars] set of chars
[(^)[char1]-[char2]] : match any chars from range [char1]-[char2]
Escape_sequances:
\A - start of the string
\b - word boundary
\B - not \b
\d - digit (0-9)
\D - not \d
\s - white space
\S - not \s
\w - word char (letters, digits or '_')
\W - not \w
\Z - end of the string
+----------------------------------------------------------+
| Simulating scanf() |
+----------------+-----------------------------------------+
| scanf() Token | Regular Expression |
+----------------+-----------------------------------------+
| %c | . |
| %5c | .{5} |
| %d | [-+]?\d+ |
| %e, %E, %f, %g | [-+]?(\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)? |
| %o | [-+]?[0-7]+ |
| %s | \S+ |
| %u | \d+ |
| %x, %X | [-+]?(0[xX])?[\dA-Fa-f]+ |
+----------------+-----------------------------------------+
match:
• always have a bool value of true
• the strings matching the parenthesised regexes are called groups in this context
• [id] is either the index number (int) of a group or its name (string)
Member_functions:
expand(<string>) : returns <string> with escape sequences converted to appropriate chars
group([id](, [id])) : returns a string or tuple of strings corresponding to int-th group or group named string
{ i = re.match(r"(\w+)", "asd a") ; print(i.groups(1)) -> "asd" }
[[id]] : identical to group()
groups() : returns a tuple to with all groups
groupdict() : returns a dictionary of with all group names - group pairs
start([id]) : returns an int representing the starting position of group [id]
end([id]) : returns an int representing the ending position of group [id]
span([id]) : returns a tuple with start([id]) and end([id])
Member_variables:
pos : ?!
endpos : ?!
lastindex : int index of the last group
lastgroup : name of the last group or None of it doesnt have a name
re : the regex object from which *this was created from
string : string from which *this was created from
Exceptions:
error(<string>, [re], [pos]) : raised when a string passed to a regex function is not a valid regex;
<string> being the error message;
[re] being the faulty regex;
[pos] being the position of [re]
~~~~~~~~~~~~~~~
socket: socket:
~~~~~~~~~~~~~~~
import socket
gethostname() : gets host user's name (pl.: kali)
socket([address family],[socket type],
[protocol number]) : creates new socket
address families:
AF_INET : IPv4
AF_INET6 : IPv6
AF_UNIX : UNIX-domain protocol family
AF_CAN : Controller Area Network
AF_PACKET : packet
AF_RDS :
socket types:
SOCK_STREAM : TCP transport protocol
SOCK_DGRAM : dgram
SOCK_RAW : raw
protocol number:
is usually zero and may be omitted or in the case where the address family
is AF_CAN the protocol should be one of CAN_RAW, CAN_BCM or CAN_ISOTP.
~~~~~~~~~~~~~~~~
requests: requests:
~~~~~~~~~~~~~~~~
import requests
functions:
[http method]([url]{, timeout = <int>}) : returns [response] class to [method] on [url];
will wait indefinitely for response unless timeout is specified; set time out!
{ get(fizika.pe.hu) }
class Response:
.text : returns source in UTF-8 (for getting html)
.content : returns source in bytes (for getting file (images))
.status_code: returns status code (optimally 200) (in 404 for example)
.headers : returns http header
.url : return url
~~~~~~~~~~~~~~~~
hashlib: hashlib:
~~~~~~~~~~~~~~~~
import hashlib
class hash:
new(<name>, <data>)
— predefined new wrappers:
md5(<data>)
sha1(<data>)
sha224(<data>)
sha256(<data>)
sha384(<data>)
sha512(<data>)
blake2b(<data>)
blake2s(<data>)
sha3_224
sha3_256
sha3_384
sha3_512
shake_128
shake_256
Methods:
update(<string>) : concat <string> to the internal data
digest() : return hash of internal data
hexdigest() : return digest() converted to a string of hex chars
{
• >> import hashlib
• >> hashlib.md5(b'nicotine').hexdigest()
'1cecdfb5a688b023ce65fd0b51fcf67f'
}
tkinter
#define tkinter:: \
I=========================================\
I=========================================\
I _____ _ _ _ \
I |_ _| | (_) | | \
I | | | | ___ _ __ | |_ ___ _ __ \
I | | | |/ / | '_ \| __/ _ \ '__| \
I | | | <| | | | | || __/ | \
I \_/ |_|\_\_|_| |_|\__\___|_| \
I=========================================\
I=========================================I
from tkinter import *
from tkinter import ttk
• "ToolKit INTERface"
• standard gui toolkit for python
• not just some standard library, it binds to Tcl/Tk
• it has to be compile time enable in the interpreter
— NOTE on memory:
• the Tk object tree internally saves references to the created objects,
therefor lifetimes work as one would expect without explicit references
• the same does not apply to say PhotoImages, they are passed to the underlying Tcl
functions, but create no copies that the python garbage collector could see,
resulting in premature destruction
{ # The BELOW display the text "Example" correctly
from tkinter import *
from tkinter import ttk
w = Tk()
Label(w, text="Example").pack()
w.mainloop()
# The BELOW fails to display the image "example.png",
# however it does sizes correctly to a blank area
from tkinter import *
from tkinter import ttk
w = Tk()
Label(w, image=PhotoImage(file="sample.png")).pack()
w.mainloop()
# The BELOW on the otherhand succeeds
from tkinter import *
from tkinter import ttk
w = Tk()
i = PhotoImage(file="example.png")
Label(w, image=i).pack()
w.mainloop()
}
Tk:
Tk : returns a top level window object
mainloop() : fire up GUI
resizable(bool, bool)
PhotoImage(file=...)
subsample(<int-h>, <int-w>) -> PhotoImage : return a shrinked copy
zoom(<int-h>, <int-w>) -> PhotoImage : return an enlarged copy
Ttk:
• ttk is part of the Tk package and provides more modern widgets based on the original ones,
python provides it as a separate (sub)module
Widgets:
Label
Button(command=<function>)
— common members:
master : parent element, always the first argument and is required
text : visible text
image
bind(<event>, <function>, <add>)
add:
'' - replace previous handlers
'+' - append to previous handlers
• widgets may be further changed using dictionary syntax { myButton["fg"] = "red" }
Events:
<[${mprefix}-]Button-${N}> Mouse button press
<B${N}-Motion> Mouse moved with the ${N}th button held down
<ButtonRelease-${N}> Mouse button realise
<Enter> Mouse hover
<Leave> Mouse hover off
<FocusIn> Widget cursor focus
<FocusOut> Widget cursor focus off
<[${prefix}-]${key}> ${key} was pressed with ${prefix} held
${key} ${key} was typed
<Configure> Size change
<Activate> Turned active
<Deactivate> Turned inactive
<Destroy> Being deleted
<Expose> Got uncovered by another window
<KeyRelease> Any key released
<Map> Getting mapped (shows up)
<Motion> Mouse move within
<MouseWheel> Wheel moved; does not work on Linux
<Unmap> Getting unmapped (gets hidden)
<Visibility> Turned visible on the screen
the outer "<>" are not my schizophrenia, they are literals required
1 is a keyboard binding, while <1> is a button binding.
Callbacks:
def ...(<element>, <event>);
Keys:
• ordinary literals { <a> }
— one of the special keys available:
Cancel
BackSpace
Tab
Return
Shift_L
Control_L
Alt_L
Pause
Caps_Lock
Escape
Prior
Next
End
Home
Left
Up
Right
Down
Print
Insert
Delete
F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
Num_Lock
Scroll_Lock
— one of the aliases
space
less
prefixes:
Alt
Control
Shift
mprefixes:
Double
Triple
Layout:
pack
side="<side>"
left
grid
place
python_
#define python_3th_party_modules:: \
I========================================================================================================\
I========================================================================================================\
I _____ _____ _ _ ______ ___ ______ _______ __ ___ ______________ _ _ _ _____ _____ \
I |____ |_ _| | | | | ___ \/ _ \ | ___ \_ _\ \ / / | \/ | _ | _ \ | | | | | ___/ ___| \
I / / | | | |_| | | |_/ / /_\ \| |_/ / | | \ V / | . . | | | | | | | | | | | | |__ \ `--. \
I \ \ | | | _ | | __/| _ || / | | \ / | |\/| | | | | | | | | | | | | __| `--. \ \
I .___/ / | | | | | | | | | | | || |\ \ | | | | | | | \ \_/ / |/ /| |_| | |____| |___/\__/ / \
I \____/ \_/ \_| |_/ \_| \_| |_/\_| \_| \_/ \_/ \_| |_/\___/|___/ \___/\_____/\____/\____/ \
I========================================================================================================\
I========================================================================================================I
~~~~~~~~~~~~~~~~~
pyautogui: pyautogui:
~~~~~~~~~~~~~~~~~
import pyautogui
Constants:
KEYBOARD_KEYS : list of [button] names
Functions:
size() : returns screen size
position() : Returns the current xy coordinates of the mouse cursor as a two-integer tuple.
click([intx], [inty], <string>) : simulates click at [intx] [inty] (in pixels), with <string> button ("left" || "right" || "middle")
mouseDown([intx], [inty], <string>) : duh; see ABOVE
mouseUp([intx], [inty], <string>) : duh; see ABOVE
typewrite(<string>) : simulates typing <string>
press([button]) : simulates [button] press
hold([button]) : simulates holding down [button]
KeyUp([button]) : simulates releasing [button]
~~~~~~~~~~~~~~
Pillow: Pillow:
~~~~~~~~~~~~~~
from PIL import Image
from PIL import ImageTk
• image processing library
• closely integrates with tkinter
Image:
open(path: string)
Image:
width
height
resize((w, h))
getpixel((x, y))
putpixel((x, y), channels: tuple)
ImageTk:
PhotoImage(Image) : return-s a converted object which can be displayed using tkinter;
it is however not fully compatible with tk PhotoImages
python_scrapping
#define python_scrapping::: \
I ___ _ \
I / __| __ _ _ __ _ _ __ _ __(_)_ _ __ _ \
I \__ \/ _| '_/ _` | '_ \ '_ \ | ' \/ _` | \
I |___/\__|_| \__,_| .__/ .__/_|_||_\__, | \
I |_| |_| |___/ I
~~~~~~~~~~~~~~~~~~~~~
beautiful_soup: beautiful_soup:
~~~~~~~~~~~~~~~~~~~~~
import bs4
BeautifulSoup(<string>, 'html.parser') : returns html (BeautifulSoup) object
[BeautifulSoup]:
.head : returns the head
.title : returns the title
.body : returns the body
.find([options]) || .find_all([options]) : returns first occurrence of [options] || returns list of all occurrences of [options]
[tag] : searches for tag {"div"}
id=[id] : searches for id
class_=[class] : searches for class
attrs={[name]:[value]} : searches for attribute
.get_text() : returns the text without any tags
.prettify() : returns source formatted nicely with tabs and spaces
Snippets:
b = BeautifulSoup(get(my_url).text, 'html.parser')
~~~~~~~~~~~~~~~~
selenium: selenium:
~~~~~~~~~~~~~~~~
modules:
webdriver : its a module inside a module, therefor you import it as follows: "from selenium import webdriver"
and you either "from" further or do webdriver.[x];
{import selenium [\n] selenium.webdriver.[...]} is invalid; (confusing ik)
functions:
[browser]() : return browser session; [browser] == "Firefox" || etc.
{ driver = webdriver.Firefox(firefox_binary="/usr/bin/firefox-esr") }
common:
actionchains : ?; import as follows: "from selenium.webdriver.common.action_chains import ActionChains"
functions:
ActionChains([browser]) : returns actionchain object
keys : keyboard keys; import it as follows: "from selenium.webdriver.common.keys import Keys"
.RETURN || .ENTER : enter
by:
By:
CLASS_NAME
CSS_SELECTOR
ID
LINK_TEXT
NAME
PARTIAL_LINK_TEXT
TAG_NAME
XPATH
[browser]:
Member_functions:
impicitly_wait(<int>) : wait <int> before timeout
get([url]) : load [url]
back() : move backward in browsers history
forward() : move forward in browsers history
find_element([By], [val])
find_elements([By], [val])
the following methods have been deprecated and replaced by the ABOVE;
which follows the exact logic a beginner, alcoholic teen (me) came up with
after a single small scale project. this proves that webdevs are mentally
handicapped and should be forced to work on the fields instead.
search:
>includeing an 's' after elements will leturn a list of results instead of the first ocurence
>returns [element]
.find_element(s|_by_id(<string>| : searches for id; {<whatever id="login">}
.find_element(s|_by_name(<string>| : searches for name; {<whatever name="login">}
.find_element(s|_by_link_text :
.find_element(s|_by_partial_link_text :
.find_element(s|_by_tag_name : serches for tag name; {<my_tag name="irrelevant">}
.find_element(s|_by_class_name :
.find_element(s|_by_css_selector : searches for class name; {if <form name="login">}
.find_element(s|_by_xpath([XPath]| : searches for XPath; can be used to search for visible text
{ driver.find_element_by_xpath('//*[text(|="text to search for"]'| }
Member_variables:
.page_source : page source
.current_url : url of current page
[element]
Member_functions:
.send_keys(<string>) : simulates writing
.click() : simulates click
.doubleClick() : simulates double clicking
.clickAndHold() : simulates clicking and holding
.getX()
.getY()
.getWidth()
.getHeight()
[actionchain]
Member_functions:
move_to_element() : simulates hoverover
perform() : actually perform actions
Hello_World:
{
from selenium import webdriver
driver = webdriver.Chrome(executable_path="/usr/bin/chromedriver")
driver.get("https://example.com")
}
~~~~~~~~~~~~~~
scrapy
~~~~~~~~~~~~~~
• OOP cancer
• slow as fuck
• tries to be both a library and a framework
• the output is unreadable
• it wraps ^C (of the shell), but it does not work
• it wraps the lxml library, but hides features (with xpath)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
google_images_download
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>outdated piece of crap
>used to allow image type filtering
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bing_image_downloader
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
• downloads images from bing
• useful for fetching machine learning datasets
from bing_image_downloader import downloader
downloader.download(
"dog",
limit=100,
output_dir="downloads/",
timeout=60
)
python_machine_learning
#define python_machine_learning::: \
I __ __ _ _ _ _ \
I | \/ |__ _ __| |_ (_)_ _ ___ | |___ __ _ _ _ _ _ (_)_ _ __ _ \
I | |\/| / _` / _| ' \| | ' \/ -_) | / -_) _` | '_| ' \| | ' \/ _` | \
I |_| |_\__,_\__|_||_|_|_||_\___| |_\___\__,_|_| |_||_|_|_||_\__, | I
~~~~~~~~~~~~~~~~~~
tensorflow
~~~~~~~~~~~~~~~~~~
import tensorflow
from tensorflow import keras
TensorFlow in Action by Thushan Ganegedara
• machine learning library
"\
tensor /tĕn′sər, -sôr″/\
noun\
A muscle that stretches or tightens a body part.\
"
• a tensor is the main abstraction of tensorflow; it is a value type that utilizes GPUs for calculations
• the low-level tensor is something one does not really have to touch due to higher level abstractions
• keras used to be a poli-backend abstraction module, however after tensorflow was left as
the last viable one, it got merged into a submodule
Files:
.ckpt : "ChecKPoinT"; model weights saved to disk
.h5 : deprecated; entire model saved to disk
.keras : entire model saved to disk
Keras:
○ has 3 main APIs
Sequential Functional Sub-classing
simple ◀━━━━━━━━━━━━━━━━━━━━━━━━━▶ complex
less powerful more powerful
• this abstract method of classification comes from "TensorFlow in Action"; (and i like it)
— Sequential
• the sub-class of Model called Sequential is parameterized
• classical feed-forward network
— Functional
• Model is parameterized
• (presumably) a less conventional architecture is used (such as one with branches)
— Sub-classing (custom)
• realistically at least "__init__", "build()" and "call()" is overridden
• requires low-level understanding of tensors and willingness to thinker with them
package_layers:
from tensorflow.keras import layers
Input(shape)
Rescale(scale, offset) : multiplies all input values by <scale> and adds offset;
used for preprocessing data, by forcing input values
in an arbitrary range
Dense(shape) : fully connected
Conv2D() : 2 dimensional convolution
MaxPool2D() : max convolution
Flatten() : convolutional-fully connected layer adapter
SimpleRNN(shape),
Concatenate(?!)
package_models:
class Model:
Members:
summary() : print pretty table regarding layers and parameters; useful for visualization
fit() : train
predict() : ask the opinion after training
persistance:
save(path) : save the entire model; the recommended extension is ".keras"
load_model(path) : load a model saved with save()
save_weights(path) : save weights only; no architectural information is retained
load_weights(path) : load weights saved with save_weights()
Subclasses:
Sequential
package_callbacks:
class EarlyStopping:
• used for creating a break condition during training
• if the model starts to overfit, fitting will terminate
constructor:
"monitors" ["loss"|"val_loss"]
"patience" <int> : number of lossy epochs to terminate after
"restore_best_weights" <bool>
preprocessing:
from tensorflow.keras import preprocessing
• used for rapidly creating datasets from common disk representations
image_dataset_from_directory(
directory,
image_size=(256, 256),
)
text_dataset_from_directory(directory)
timeseries_dataset_from_array(directory)
### ACTUALLY simple Tensorflow example ###
#!/bin/python3
import numpy as np
from tensorflow import keras
# ---------------
# --- Dataset ---
# ---------------
#NOTE: this is where i do NOT ask you to download 5GBs of samples
def gen_data():
# Addition in the finite field of 0..9
r = {'in': [], 'out': []}
for i in range(10):
for h in range(10):
r['in'].append((i, h))
r['out'].append((i + h) % 10)
r['in'] = np.array(r['in']) # tensorflow does not accept python lists
r['out'] = np.array(r['out'])
return r
dataset = gen_data()
# -------------
# --- Model ---
# -------------
model = keras.Sequential() # Stock feedforward network
hidden_layers = [2, 8, 4, 10, 8] # Overkill is the best kind of kill
for i in hidden_layers: model.add(keras.layers.Dense(i, activation='relu'))
model.add(keras.layers.Dense(1)) # output layer
model.compile(
optimizer='adam',
loss='mse', # Mean Square error - for calculating how wrong the model was (margins will grow exponentially)
metrics=['accuracy']
)
# Training
model.fit(dataset['in'], dataset['out'],
verbose=2, # max level of output during training
batch_size=10,
epochs=5000, # Repetition count on the whole dataset; again, overkill
shuffle=True,
)
# ------------------------------
# --- Interactive playground ---
# ------------------------------
#NOTE: importing will work too
def main():
while True:
try:
a = int(input("Enter the first integer (a): "))
b = int(input("Enter the second integer (b): "))
r1 = model.predict(np.array([(a, b)]))[0][0]
r2 = np.round(r1)
print(f"The sum of {a} and {b} is {r2} ({r1})")
except ValueError:
pass
if __name__ == '__main__':
main()
# Now try playing around with the variables
#
### Basic image classification example ###
— set up all the data we need:
• one black image
• one white image
#!/bin/bash
mkdir dataset
mkdir dataset/white
mkdir dataset/black
convert -size 100x100 xc:white dataset/white/white.png
convert -size 100x100 xc:black dataset/black/black.png
• make our model
#!/bin/python3
from sys import argv
from tensorflow import keras
my_activation = ( # declared like this to ease commenting/uncommenting
#'sigmoid'
# performs like absolute trash
# requires ~x4 more epochs than relu
#'relu'
# has the tendency to produce such probabilities:
# white.png - 0.00% black : 100.00% white
# black.png - 51.10% black : 48.90% white
# which is not surprising considering its inherent asymmetry
# requires roughly 50 epochs and slight luck
'tanh'
# easily adjusts under 10 epochs
# produces reasonable divided probabilites
)
HEIGHT, WIDTH = 20, 20
dataset = keras.utils.image_dataset_from_directory(
"dataset/",
image_size=(HEIGHT, WIDTH),
)
model = keras.Sequential([
# normally we would use convolutional layers
# before flattening and adding a few dense layers,
# however the example is so simple we dont care about spacial information
keras.layers.Flatten(),
keras.layers.Dense(8, activation=my_activation),
keras.layers.Dense(8, activation=my_activation),
keras.layers.Dense(1, activation='sigmoid')
])
model.compile(
'adam',
loss='binary_crossentropy',
metrics=['accuracy']
)
model.fit(dataset, epochs=10)
img = keras.preprocessing.image.load_img(argv[1], target_size=(HEIGHT, WIDTH))
img = keras.utils.img_to_array(img)
img = keras.ops.expand_dims(img, 0)
score = model.predict(img)[0][0]
print(f"{100 * (1 - score):.2f}% black : {100 * score:.2f}% white")
#
— if the import cries about your GPU and such,
you can silence like this:
import os
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"
~~~~~~~~~~~~~~~~~~~~
transformers: transformers:
~~~~~~~~~~~~~~~~~~~~
• closely associated with https://huggingface.co/
• highlevel machine learning framework
class pipeline:
• functor like
constructor(<string-type>)
text-classification : determines sentiment
text-generation : attempts to continue the input string
zero-shot-classification : classify text into user provided candidate categories
question-answering : returns where the answer is in an input text to an input question;
NOT a higher-level text generation model;
the full answer being in multiple places confuses it
import transformers
print(transformers.pipeline("sentiment-analysis")("I am very angry"))
# [{'label': 'NEGATIVE', 'score': 0.9994483891868591}]
~~~~~~~~~~~~~~
petals: petals:
~~~~~~~~~~~~~~
• p2p LLM library
• allows for treating remote models as local
• has a network of (GPU) volunteers backing the project
from transformers import AutoTokenizer
from petals import AutoDistributedModelForCausalLM
# Choose any model available at https://health.petals.dev
model_name = "petals-team/StableBeluga2"
# Connect to a distributed network hosting model layers
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoDistributedModelForCausalLM.from_pretrained(model_name)
# Run the model as if it were on your computer
inputs = tokenizer("A cat sat", return_tensors="pt")["input_ids"]
outputs = model.generate(inputs, max_new_tokens=5)
print(tokenizer.decode(outputs[0])) # A cat sat on a mat...
brython
#define brython:: \
I---------------------------------------------------------------------------------\
I \
I ███████████ █████ █████ \
I ░░███░░░░░███ ░░███ ░░███ \
I ░███ ░███ ████████ █████ ████ ███████ ░███████ ██████ ████████ \
I ░██████████ ░░███░░███░░███ ░███ ░░░███░ ░███░░███ ███░░███░░███░░███ \
I ░███░░░░░███ ░███ ░░░ ░███ ░███ ░███ ░███ ░███ ░███ ░███ ░███ ░███ \
I ░███ ░███ ░███ ░███ ░███ ░███ ███ ░███ ░███ ░███ ░███ ░███ ░███ \
I ███████████ █████ ░░███████ ░░█████ ████ █████░░██████ ████ █████ \
I ░░░░░░░░░░░ ░░░░░ ░░░░░███ ░░░░░ ░░░░ ░░░░░ ░░░░░░ ░░░░ ░░░░░ \
I ███ ░███ \
I ░░██████ \
I ░░░░░░ \
I---------------------------------------------------------------------------------I
• python implementation in Javascript
• allows for user-side python seamlessly interacting with the DOM
• its kinda slow (who would have guessed that python is slow?)
• whitespace sensitivity means minimizing it is not possible like with Javascript
Programs:
brython-cli <options> <verb>
• used for creating local installations
<verb>
install : creates a local copy of the interpreter files,
to avoid external dependencies
Dependencies:
• since the interpreter is written in javascript, it has to be included somewhere
brython.js : the core dependency for the language to work
{
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/brython/3.10.0/brython.min.js"></script>
<script type="text/javascript" src="/brython.min.js"></script>
}
brython_stdlib.js : the python standard library (reimplemented in javascript,
exposed to brython natively)
{
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/brython/3.10.0/brython_stdlib.min.js"></script>
<script type="text/javascript" src="/brython_stdlib.min.js"></script>
}
Embeding:
• brython code is placed into its own script tag, appropriately marked
{
<script src="https://www.example.com/main.py" type="text/python"></script>
<script type="text/python">
print("Hello World!")
</script>
}
• brython tags must be explicitly invoked using Javascript somehow
{
<script type="text/javascript">
brython({});
</script>
}
• modules imported must be served somehow, dynamic serving is the default,
however the used modules can be prepackaged to the app
{
$ brython-cli --modules # this will generate a "modules" file, which can be included in one
<script type="text/javascript" src="brython_modules.js"></script>
}
Modules:
#================================================================================#
I Key browser API modules I
#==================#==============================#==============================#
I Modules | Context | Examples I
#==================#==============================#==============================#
| browser | Contains the built-in names | browser.alert() |
| | and modules | |
+------------------+------------------------------+------------------------------+
| browser.document | Accesses the DOM | document["element-id"] |
+------------------+------------------------------+------------------------------+
| browser.html | Creates HTML elements | html.H1("This is the title") |
+------------------+------------------------------+------------------------------+
| browser.window | Accesses Window functions | window.navigator |
| | and objects | |
+------------------+------------------------------+------------------------------+
| javascript | Accesses objects defined | javascript.JSON.parse() |
| | in JavaScript | |
#==================#==============================#==============================#