torchfun package

Submodules

torchfun.datasets module

class torchfun.datasets.ImageAugmentationDataset(root, pre_transform=None, degrade_transform=None, post_transform=None, loader=<function default_loader>)

Bases: torchvision.datasets.folder.ImageFolder

Pre_transform is applied to the source image firstly. Then, degrade_transform is applied to get degraded image (such as blurred image). Finally, both the degraded imgs and the pre-transformed images are uniformly post-transformed. Usually, the post-transforms are ToTensor() and Normalize()

Degrade transform should have two arguments as input:

1: input image to be degraded 2: Boolean input indicates if the degrading parameters should be returned.

torchfun.nn module

Neural Network related layers/functions/classes that are compatible with all pyTorch usage.

class torchfun.nn.AbsMax(dim=1)

Bases: torch.nn.modules.module.Module

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class torchfun.nn.Conv2dDepthShared(in_channels, out_channels, trunks, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True)

Bases: torch.nn.modules.conv.Conv2d

Applies a 2D convolution over an input signal composed of several input planes.

Share the kernel along depth/channel direction. Conv2dDepthShared divides input images into multiple sub-layers(trunks) along depth axis, and use shared kernel to process each depth trunk.

In the simplest case, the output value of the layer with input size \((N, C_{in}, H, W)\) and output \((N, C_{out}, H_{out}, W_{out})\) can be precisely described as:

\[\begin{equation*} \text{out}(N_i, C_{out_j}) = \text{bias}(C_{out_j}) + \sum_{k = 0}^{C_{in} - 1} \text{weight}(C_{out_j}, k) \star \text{input}(N_i, k) \end{equation*},\]

where \(\star\) is the valid 2D cross-correlation operator, \(N\) is a batch size, \(C\) denotes a number of channels, \(H\) is a height of input planes in pixels, and \(W\) is width in pixels. The weight and bias matrix of a Conv2dDepthShared are low-rank. They share trunks of digits repeatitively inside their matrices.

Example:

Conv(in=3,out=9,k=3,s=1) will create kernel weight with size of (9x3x5x5) kernel of a Depth-shared-Conv2d only has 3x1x5x5 parameters.

• stride controls the stride for the cross-correlation, a single number or a tuple.

• padding controls the amount of implicit zero-paddings on both sides for padding number of points for each dimension.

• dilation controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this link has a nice visualization of what dilation does.

• groups controls the connections between inputs and outputs. in_channels and out_channels must both be divisible by groups. For example,

  • At groups=1, all inputs are convolved to all outputs.
  • At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated.
  • At groups= in_channels, each input channel is convolved with its own set of filters (of size \(\left\lfloor\frac{\text{out_channels}}{\text{in_channels}}\right\rfloor\)).

The parameters kernel_size, stride, padding, dilation can either be:

• a single int – in which case the same value is used for the height and width dimension
• a tuple of two ints – in which case, the first int is used for the height dimension, and the second int for the width dimension

Note

Depending of the size of your kernel, several (of the last) columns of the input might be lost, because it is a valid cross-correlation, and not a full cross-correlation. It is up to the user to add proper padding.

Note

The configuration when groups == in_channels and out_channels == K * in_channels where K is a positive integer is termed in literature as depthwise convolution.

In other words, for an input of size \((N, C_{in}, H_{in}, W_{in})\), if you want a depthwise convolution with a depthwise multiplier K, then you use the constructor arguments \((\text{in_channels}=C_{in}, \text{out_channels}=C_{in} * K, ..., \text{groups}=C_{in})\)

Args:

in_channels (int): Number of channels in the input image, inchannels must can be divided by trunks. out_channels (int): Number of channels produced by the convolution. out channels must can be divided by trunks. trunks (int): Number of trunks a single image is divided into (along depth). All trunks inside an image share same weight/bias. kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): Zero-padding added to both sides of the input. Default: 0 dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If True, adds a learnable bias to the output. Default: True

Shape:

• Input: \((N, C_{in}, H_{in}, W_{in})\)

• Output: \((N, C_{out}, H_{out}, W_{out})\) where

  \[ \begin{align}\begin{aligned}H_{out} = \left\lfloor\frac{H_{in} + 2 * \text{padding}[0] - \text{dilation}[0] * (\text{kernel_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor\\W_{out} = \left\lfloor\frac{W_{in} + 2 * \text{padding}[1] - \text{dilation}[1] * (\text{kernel_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor\end{aligned}\end{align} \]

Attributes:

weight (Tensor): the learnable weights of the module of shape

(out_channels, in_channels, kernel_size[0], kernel_size[1])

bias (Tensor): the learnable bias of the module of shape (out_channels)

Examples:

>>> # With square kernels and equal stride
>>> m = nn.Conv2d(16, 33, 3, stride=2)
>>> # non-square kernels and unequal stride and with padding
>>> m = nn.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2))
>>> # non-square kernels and unequal stride and with padding and dilation
>>> m = nn.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2), dilation=(3, 1))
>>> input = torch.randn(20, 16, 50, 100)
>>> output = m(input)

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class torchfun.nn.Flatten

Bases: torch.nn.modules.module.Module

Flatten module Usage:

flat = Flatten() out = flat(x)

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class torchfun.nn.Squeeze(dim=None)

Bases: torch.nn.modules.module.Module

squeeze(input, dim=None, out=None) -> Tensor

Returns a tensor with all the dimensions of input of size 1 removed.

For example, if input is of shape: \((A imes 1 imes B imes C imes 1 imes D)\) then the out tensor will be of shape: \((A imes B imes C imes D)\).

When dim is given, a squeeze operation is done only in the given dimension. If input is of shape: \((A imes 1 imes B)\), squeeze(input, 0) leaves the tensor unchanged, but squeeze(input, 1)() will squeeze the tensor to the shape \((A imes B)\).

Note

As an exception to the above, a 1-dimensional tensor of size 1 will not have its dimensions changed.

Note

The returned tensor shares the storage with the input tensor, so changing the contents of one will change the contents of the other.

Example:

>>> x = torch.zeros(2, 1, 2, 1, 2)
>>> x.size()
torch.Size([2, 1, 2, 1, 2])
>>> y = torch.squeeze(x)
>>> y.size()
torch.Size([2, 2, 2])
>>> y = torch.squeeze(x, 0)
>>> y.size()
torch.Size([2, 1, 2, 1, 2])
>>> y = torch.squeeze(x, 1)
>>> y.size()
torch.Size([2, 2, 1, 2]) 

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class torchfun.nn.Subpixel(out_channels=1, stride=None)

Bases: torch.nn.modules.module.Module

Unfold channel/depth dimensions to enlarge the feature map Notice: Output size is deducted. The size of the unfold square is automatically determined e.g. :

images: 100x16x16x9. 9=3x3 square subpixel-out: 100x48x48x1

Arguement:

out_channels, channel number of output feature map stride: enlarging ratio of spatial dimensions. stride=2 outputs x4 img area. If provided, out_channels will be ignored.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

torchfun.nn.flatten(x)

Flatten function Usage:

out = flatten(x)

torchfun.nn.subpixel(x, out_channels=1)

Unfold channel/depth dimensions to enlarge the feature map Notice: Output size is deducted. The size of the unfold square is automatically determined e.g. :

images: 100x9x16x16. 9=3x3 square subpixel-out: 100x1x48x48

Arguement:

x: NCHW image, channel first. out_channels, channel number of output feature map

torchfun.torchfun module

class torchfun.torchfun.Packsearch(module_object, auto_init=True, verbose=False)

Bases: object

Given an module object as input: > p = Packsearch(torch) or > p = Packsearch(‘torch’) the instance p provide p.search() method. So that you can search everything inside this package > p.search(‘maxpoo’) or simply > p(‘maxpoo’) output:

Packsearch: 35 results found: ————-results start————- 0 torch.nn.AdaptiveMaxPool1d 1 torch.nn.AdaptiveMaxPool2d 2 torch.nn.AdaptiveMaxPool3d 3 torch.nn.FractionalMaxPool2d 4 torch.nn.MaxPool1d 5 torch.nn.MaxPool2d …

dynamic_traverse(mod, query)

traverse the module and simultaneously search for queried name

preprocess_names()

search(name)

traverse(mod, search_attributes=False)

gather all names and store them into a name_list search_attributes: whether to include class attributes or method names

torchfun.torchfun.count_parameters(model_or_dict_or_param)

Count parameter numer of a module/state_dict/layer/tensor. This function can also print the occupied memory of parameters in MBs Arguements: model_or_dict_or_param: model or state dictionary or model.parameter(), or numpy-array, or tensor. Return: parameter amount in python-int

Returns 0 if datatype not understood

Usage: 1. count trainable and untrainbale params

count_parameters(model)

same as

count_parameters(state_dict)

2. count only trainable params:

   count_parameters(model.parameters())

3. count data matrix

   count_parameters(weight_tensor) count_parameters(numpy_array)

Alias: parameters()

torchfun.torchfun.force_exist(dirname, verbose=True)

force a directory to exist. force_exist can automatically create directory with any depth. Arguements:

dirname: path of the desired directory verbose: print every directory creation. default True.

Usage:

force_exist(‘a/b/c/d/e/f’) force_exist(‘a/b/c/d/e/f’,verbose=False)

torchfun.torchfun.hash_parameters(model_or_statdict_or_param, use_sum=False)

return the summary of all variables. This is used to detect chaotic changes of weights. You can check the sum_parameters before and after some operations, to know if there is any change made to the params.

I use this function to verify gradient behaviours.

By default, This only hash the trainable parameters!

arguements: module_or_statdict_or_param: torch.nn.module,

or model.state_dict(), or model.parameters().

use_sum: return the sum instead of mean value of all params.

torchfun.torchfun.imshow(x, title=None, auto_close=True)

only deal with torch channel-first image batch, title: add title to plot. (Default None)

title can be string, or any string-able object.

auto_close: (default True)

Close the pyplot session afterwards. Clean the environment just like you had never used matplotlib here. if set to False, the plot will remain in the memory for further drawings.

torchfun.torchfun.load(a, b)

Load weight a into model b, or load model b using weight a The order of the arguments doesn’t matter. Example:

>load(‘weights.pts’,model)

or

>load(model,’weights.pts’)

or

>f = open(‘weight.pts’) >load(f,model)

or

>load(model,f)

Return value: None

torchfun.torchfun.omini_open(path)

torchfun.torchfun.packsearch(module_or_str, str_or_module)

Given an module object, and search pattern string as input: > packsearch(torch,’maxpoo’) or > packsearch(‘maxpoo’,torch) you can search everything inside this package output:

Packsearch: 35 results found: ————-results start————- 0 torch.nn.AdaptiveMaxPool1d 1 torch.nn.AdaptiveMaxPool2d 2 torch.nn.AdaptiveMaxPool3d 3 torch.nn.FractionalMaxPool2d 4 torch.nn.MaxPool1d 5 torch.nn.MaxPool2d …

torchfun.torchfun.parameters(model_or_dict_or_param)

Count parameter numer of a module/state_dict/layer/tensor. This function can also print the occupied memory of parameters in MBs Arguements: model_or_dict_or_param: model or state dictionary or model.parameter(), or numpy-array, or tensor. Return: parameter amount in python-int

Returns 0 if datatype not understood

Usage: 1. count trainable and untrainbale params

count_parameters(model)

same as

count_parameters(state_dict)

2. count only trainable params:

   count_parameters(model.parameters())

3. count data matrix

   count_parameters(weight_tensor) count_parameters(numpy_array)

Alias: parameters()

torchfun.torchfun.pil_imshow(arr)

Simple showing of an image through an external viewer.

This function is only available if Python Imaging Library (PIL) is installed.

Uses the image viewer specified by the environment variable SCIPY_PIL_IMAGE_VIEWER, or if that is not defined then see, to view a temporary file generated from array data.

Warning

This function uses bytescale under the hood to rescale images to use the full (0, 255) range if mode is one of None, 'L', 'P', 'l'. It will also cast data for 2-D images to uint32 for mode=None (which is the default).

arr : ndarray

Array of image data to show.

None

>>> a = np.tile(np.arange(255), (255,1))
>>> from scipy import misc
>>> misc.imshow(a)

Ported and upgraded based on scipy.misc.imshow Open-sourced according to the license.

torchfun.torchfun.save(a, b)

save weight a into target b, or save model b into target a The order of the arguments doesn’t matter. Example:

>save(‘weights.pts’,model)

or

>save(model,’weights.pts’)

or

>f = open(‘weight.pts’) >save(f,model)

or

>save(model,f)

or

>save(‘weights.pts’,state_dict)

or

>save(state_dict,’weights.pts’)

Return value: None

torchfun.torchfun.show_layers_parameters(model)

torchfun.torchfun.sort_args(args_or_types, types_or_args)

This is a very interesting function. It is used to support __arbitrary-arguments-ordering__ in TorchFun.

Input:

The function takes a list of types, and a list of arguments.

Returns:

a list of arguments, with the same order as the types-list.

Of course, sort_args supports arbitrary-arguments-ordering by itself.

torchfun.torchfun.tf_session(allow_growth=True)

Used to create tensorflow session that does not stupidly and unresonably consume all gpu-memeory. returns:

a tensorflow session consuming dynamic gpu memory.

torchfun.torchfun.whereis(module_or_string, open_gui=True)

find the source file location of a module arguments:

module_or_string: target module object, or it’s string path like torch.nn open_gui: open the folder with default window-manager.

returns:

module file name, or None

torchfun.transforms module

Transforms used for data augmentation in torchvision.datasets.* or in torch.utils.data.Dataset

class torchfun.transforms.RandomGaussianBlur(kernel_ratio=0.01, random_ratio=0.005, pixel_range=None)

Bases: object

PIL image

Module contents

TorchFun project was initiated long ago and was published in 2018-6-13.

TorchFun is motivated by the one author loves, who sometimes encounter inconvenience using PyTorch at her work. The author published the project as a python package TorchFun on PyPi, so that his beloved could access it whenever and wheresoever needed.

The purposed of TorchFun is to provide functions which are important and convenient, but absent in PyTorch, like some layers and visualization utils.

This project has been undergoing in secret so that the author could credit TorchFun to his beloved as a little supprise of help, one day when this project is well-enriched or gets famous.

Interestingly, The original project name given by the author, is Torchure. That is because he was always multi-tasking trivial affairs in school and got scorched, and when he was learning this new framework in hope to help his beloved, he found plenty of issues/missing-functionalities. He felt that this was totally a torture. So, this project was named “Torchure” to satirize the lame PyTorch (you can still found Torchure in PyPi). And, the author hoped, by developing Torchure, his beloved could feel ease even when encountering the crappy-parts of PyTorch.

This history-of-project was appended recently, because his adorable little beloved wants a supprise immediately, or she will keep rolling on the floor.

To c71b05bf46d8772e4488335085a2e7fd.

torchfun.subpixel(x, out_channels=1)

Unfold channel/depth dimensions to enlarge the feature map Notice: Output size is deducted. The size of the unfold square is automatically determined e.g. :

images: 100x9x16x16. 9=3x3 square subpixel-out: 100x1x48x48

Arguement:

x: NCHW image, channel first. out_channels, channel number of output feature map

torchfun.count_parameters(model_or_dict_or_param)

Count parameter numer of a module/state_dict/layer/tensor. This function can also print the occupied memory of parameters in MBs Arguements: model_or_dict_or_param: model or state dictionary or model.parameter(), or numpy-array, or tensor. Return: parameter amount in python-int

Returns 0 if datatype not understood

Usage: 1. count trainable and untrainbale params

count_parameters(model)

same as

count_parameters(state_dict)

2. count only trainable params:

   count_parameters(model.parameters())

3. count data matrix

   count_parameters(weight_tensor) count_parameters(numpy_array)

Alias: parameters()

class torchfun.Packsearch(module_object, auto_init=True, verbose=False)

Bases: object

Given an module object as input: > p = Packsearch(torch) or > p = Packsearch(‘torch’) the instance p provide p.search() method. So that you can search everything inside this package > p.search(‘maxpoo’) or simply > p(‘maxpoo’) output:

Packsearch: 35 results found: ————-results start————- 0 torch.nn.AdaptiveMaxPool1d 1 torch.nn.AdaptiveMaxPool2d 2 torch.nn.AdaptiveMaxPool3d 3 torch.nn.FractionalMaxPool2d 4 torch.nn.MaxPool1d 5 torch.nn.MaxPool2d …

dynamic_traverse(mod, query)

traverse the module and simultaneously search for queried name

preprocess_names()

search(name)

traverse(mod, search_attributes=False)

gather all names and store them into a name_list search_attributes: whether to include class attributes or method names

class torchfun.Squeeze(dim=None)

Bases: torch.nn.modules.module.Module

squeeze(input, dim=None, out=None) -> Tensor

Returns a tensor with all the dimensions of input of size 1 removed.

For example, if input is of shape: \((A imes 1 imes B imes C imes 1 imes D)\) then the out tensor will be of shape: \((A imes B imes C imes D)\).

When dim is given, a squeeze operation is done only in the given dimension. If input is of shape: \((A imes 1 imes B)\), squeeze(input, 0) leaves the tensor unchanged, but squeeze(input, 1)() will squeeze the tensor to the shape \((A imes B)\).

Note

As an exception to the above, a 1-dimensional tensor of size 1 will not have its dimensions changed.

Note

The returned tensor shares the storage with the input tensor, so changing the contents of one will change the contents of the other.

Example:

>>> x = torch.zeros(2, 1, 2, 1, 2)
>>> x.size()
torch.Size([2, 1, 2, 1, 2])
>>> y = torch.squeeze(x)
>>> y.size()
torch.Size([2, 2, 2])
>>> y = torch.squeeze(x, 0)
>>> y.size()
torch.Size([2, 1, 2, 1, 2])
>>> y = torch.squeeze(x, 1)
>>> y.size()
torch.Size([2, 2, 1, 2]) 

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class torchfun.AbsMax(dim=1)

Bases: torch.nn.modules.module.Module

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

torchfun.packsearch(module_or_str, str_or_module)

Given an module object, and search pattern string as input: > packsearch(torch,’maxpoo’) or > packsearch(‘maxpoo’,torch) you can search everything inside this package output:

Packsearch: 35 results found: ————-results start————- 0 torch.nn.AdaptiveMaxPool1d 1 torch.nn.AdaptiveMaxPool2d 2 torch.nn.AdaptiveMaxPool3d 3 torch.nn.FractionalMaxPool2d 4 torch.nn.MaxPool1d 5 torch.nn.MaxPool2d …

class torchfun.Conv2dDepthShared(in_channels, out_channels, trunks, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True)

Bases: torch.nn.modules.conv.Conv2d

Applies a 2D convolution over an input signal composed of several input planes.

Share the kernel along depth/channel direction. Conv2dDepthShared divides input images into multiple sub-layers(trunks) along depth axis, and use shared kernel to process each depth trunk.

In the simplest case, the output value of the layer with input size \((N, C_{in}, H, W)\) and output \((N, C_{out}, H_{out}, W_{out})\) can be precisely described as:

\[\begin{equation*} \text{out}(N_i, C_{out_j}) = \text{bias}(C_{out_j}) + \sum_{k = 0}^{C_{in} - 1} \text{weight}(C_{out_j}, k) \star \text{input}(N_i, k) \end{equation*},\]

where \(\star\) is the valid 2D cross-correlation operator, \(N\) is a batch size, \(C\) denotes a number of channels, \(H\) is a height of input planes in pixels, and \(W\) is width in pixels. The weight and bias matrix of a Conv2dDepthShared are low-rank. They share trunks of digits repeatitively inside their matrices.

Example:

Conv(in=3,out=9,k=3,s=1) will create kernel weight with size of (9x3x5x5) kernel of a Depth-shared-Conv2d only has 3x1x5x5 parameters.

• stride controls the stride for the cross-correlation, a single number or a tuple.

• padding controls the amount of implicit zero-paddings on both sides for padding number of points for each dimension.

• dilation controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this link has a nice visualization of what dilation does.

• groups controls the connections between inputs and outputs. in_channels and out_channels must both be divisible by groups. For example,

  • At groups=1, all inputs are convolved to all outputs.
  • At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated.
  • At groups= in_channels, each input channel is convolved with its own set of filters (of size \(\left\lfloor\frac{\text{out_channels}}{\text{in_channels}}\right\rfloor\)).

The parameters kernel_size, stride, padding, dilation can either be:

• a single int – in which case the same value is used for the height and width dimension
• a tuple of two ints – in which case, the first int is used for the height dimension, and the second int for the width dimension

Note

Depending of the size of your kernel, several (of the last) columns of the input might be lost, because it is a valid cross-correlation, and not a full cross-correlation. It is up to the user to add proper padding.

Note

The configuration when groups == in_channels and out_channels == K * in_channels where K is a positive integer is termed in literature as depthwise convolution.

In other words, for an input of size \((N, C_{in}, H_{in}, W_{in})\), if you want a depthwise convolution with a depthwise multiplier K, then you use the constructor arguments \((\text{in_channels}=C_{in}, \text{out_channels}=C_{in} * K, ..., \text{groups}=C_{in})\)

Args:

in_channels (int): Number of channels in the input image, inchannels must can be divided by trunks. out_channels (int): Number of channels produced by the convolution. out channels must can be divided by trunks. trunks (int): Number of trunks a single image is divided into (along depth). All trunks inside an image share same weight/bias. kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): Zero-padding added to both sides of the input. Default: 0 dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If True, adds a learnable bias to the output. Default: True

Shape:

• Input: \((N, C_{in}, H_{in}, W_{in})\)

• Output: \((N, C_{out}, H_{out}, W_{out})\) where

  \[ \begin{align}\begin{aligned}H_{out} = \left\lfloor\frac{H_{in} + 2 * \text{padding}[0] - \text{dilation}[0] * (\text{kernel_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor\\W_{out} = \left\lfloor\frac{W_{in} + 2 * \text{padding}[1] - \text{dilation}[1] * (\text{kernel_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor\end{aligned}\end{align} \]

Attributes:

weight (Tensor): the learnable weights of the module of shape

(out_channels, in_channels, kernel_size[0], kernel_size[1])

bias (Tensor): the learnable bias of the module of shape (out_channels)

Examples:

>>> # With square kernels and equal stride
>>> m = nn.Conv2d(16, 33, 3, stride=2)
>>> # non-square kernels and unequal stride and with padding
>>> m = nn.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2))
>>> # non-square kernels and unequal stride and with padding and dilation
>>> m = nn.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2), dilation=(3, 1))
>>> input = torch.randn(20, 16, 50, 100)
>>> output = m(input)

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

torchfun.sort_args(args_or_types, types_or_args)

This is a very interesting function. It is used to support __arbitrary-arguments-ordering__ in TorchFun.

Input:

The function takes a list of types, and a list of arguments.

Returns:

a list of arguments, with the same order as the types-list.

Of course, sort_args supports arbitrary-arguments-ordering by itself.

torchfun.whereis(module_or_string, open_gui=True)

find the source file location of a module arguments:

module_or_string: target module object, or it’s string path like torch.nn open_gui: open the folder with default window-manager.

returns:

module file name, or None

torchfun.force_exist(dirname, verbose=True)

force a directory to exist. force_exist can automatically create directory with any depth. Arguements:

dirname: path of the desired directory verbose: print every directory creation. default True.

Usage:

force_exist(‘a/b/c/d/e/f’) force_exist(‘a/b/c/d/e/f’,verbose=False)

torchfun.save(a, b)

save weight a into target b, or save model b into target a The order of the arguments doesn’t matter. Example:

>save(‘weights.pts’,model)

or

>save(model,’weights.pts’)

or

>f = open(‘weight.pts’) >save(f,model)

or

>save(model,f)

or

>save(‘weights.pts’,state_dict)

or

>save(state_dict,’weights.pts’)

Return value: None

torchfun.imshow(x, title=None, auto_close=True)

only deal with torch channel-first image batch, title: add title to plot. (Default None)

title can be string, or any string-able object.

auto_close: (default True)

Close the pyplot session afterwards. Clean the environment just like you had never used matplotlib here. if set to False, the plot will remain in the memory for further drawings.

torchfun.flatten(x)

Flatten function Usage:

out = flatten(x)

torchfun.generator_type

alias of builtins.generator

class torchfun.Flatten

Bases: torch.nn.modules.module.Module

Flatten module Usage:

flat = Flatten() out = flat(x)

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class torchfun.RandomGaussianBlur(kernel_ratio=0.01, random_ratio=0.005, pixel_range=None)

Bases: object

PIL image

torchfun.load(a, b)

Load weight a into model b, or load model b using weight a The order of the arguments doesn’t matter. Example:

>load(‘weights.pts’,model)

or

>load(model,’weights.pts’)

or

>f = open(‘weight.pts’) >load(f,model)

or

>load(model,f)

Return value: None

torchfun.hash_parameters(model_or_statdict_or_param, use_sum=False)

return the summary of all variables. This is used to detect chaotic changes of weights. You can check the sum_parameters before and after some operations, to know if there is any change made to the params.

I use this function to verify gradient behaviours.

By default, This only hash the trainable parameters!

arguements: module_or_statdict_or_param: torch.nn.module,

or model.state_dict(), or model.parameters().

use_sum: return the sum instead of mean value of all params.

torchfun.pil_imshow(arr)

Simple showing of an image through an external viewer.

This function is only available if Python Imaging Library (PIL) is installed.

Uses the image viewer specified by the environment variable SCIPY_PIL_IMAGE_VIEWER, or if that is not defined then see, to view a temporary file generated from array data.

Warning

This function uses bytescale under the hood to rescale images to use the full (0, 255) range if mode is one of None, 'L', 'P', 'l'. It will also cast data for 2-D images to uint32 for mode=None (which is the default).

arr : ndarray

Array of image data to show.

None

>>> a = np.tile(np.arange(255), (255,1))
>>> from scipy import misc
>>> misc.imshow(a)

Ported and upgraded based on scipy.misc.imshow Open-sourced according to the license.

class torchfun.Subpixel(out_channels=1, stride=None)

Bases: torch.nn.modules.module.Module

Unfold channel/depth dimensions to enlarge the feature map Notice: Output size is deducted. The size of the unfold square is automatically determined e.g. :

images: 100x16x16x9. 9=3x3 square subpixel-out: 100x48x48x1

Arguement:

out_channels, channel number of output feature map stride: enlarging ratio of spatial dimensions. stride=2 outputs x4 img area. If provided, out_channels will be ignored.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

torchfun.parameters(model_or_dict_or_param)

Count parameter numer of a module/state_dict/layer/tensor. This function can also print the occupied memory of parameters in MBs Arguements: model_or_dict_or_param: model or state dictionary or model.parameter(), or numpy-array, or tensor. Return: parameter amount in python-int

Returns 0 if datatype not understood

Usage: 1. count trainable and untrainbale params

count_parameters(model)

same as

count_parameters(state_dict)

2. count only trainable params:

   count_parameters(model.parameters())

3. count data matrix

   count_parameters(weight_tensor) count_parameters(numpy_array)

Alias: parameters()

torchfun.tf_session(allow_growth=True)

Used to create tensorflow session that does not stupidly and unresonably consume all gpu-memeory. returns:

a tensorflow session consuming dynamic gpu memory.

torchfun.omini_open(path)

torchfun.show_layers_parameters(model)

torchfun.module_type

alias of builtins.module