All Papers

Table of Contents

1. Summary

Summary With the recent success of neural seq2seq models for text to SQL translation, some questions are raised in regards how models generalize with unseen data. The authors diagnose the bottleneck and propose a new testbed. Additionally the authors designed a simple but effective auxil-iary task, which serves as a supportive modelas well as a regularization term to the gener-ation task to increase the models generalization. Figure 3: Illustration of our model. The upper figure is the text-to-SQL generation model which consists of threeparts: encoder (lower left), AGG/SEL decoder (upper left) and where decoder...

Table of Contents

1. Summary
2. Differences to input Normalization
3. Batch Normalization for CNN and other networks
4. Accelerating a BN Network

Summary Neural Network are generally hard to train. Changing the parameters of one layer changes the input distribution for the following layer. This change in distribution is called a covariant shift. To reduce this needed adaptation of the layer to the covariant shift, the authors present “Batch Normalization” (BN). This method normalizes the data for each mini-batch, between the layers. This allows to use a higher learning rate, be less careful about the initialization and can act as regularizer....

Table of Contents

1. Summary

Summary In this article the authors describe how AI has impact on the development of software. AI can help in the development of software in multiple points. According to the article: The new tools can be used to help reduce keystrokes, detect bugs as software is being written and automate many of the tests needed to confirm the quality of software. In an article published by Delloite authors David Schatsky and Sourabh Bumb claim that AI can augment human in the software development area instead of replacing them. Additionally Schatsky claimes: Many companies that...

Table of Contents

1. Summarize
2. Network
3. Data augmentation

Summarize The U-Net is a Convolutional network consisting of two paths, called contracting and expanding path. This network was developed for the use of medical image segmentation. It learns fast and does not need a lot of images to perform better then the state-of-the-art networks of this time. It heavily used data augmentation to artificially increase the number of available images. This network won the EM segmentation challange at ISBI 2012 by a large margin. Network The network consisting of the two paths (contracting and expanding) is build on the architecture...

Table of Contents

1. Summary
2. Network
   1. Structure
   2. Preventing Overfitting
      1. Data Augmentation
      2. Dropout

Summary Alex Krizhevsky et al. created a deep convolutional neural network, that outperformed other methods of image classification from the ImageNet LSVRC-2010 contest. With a modified variant with this model they entered the ILSVRC-2012 competition and won by a significant margin. The network classified 1.2 million images with 1000 different classes. The architecture was a simple network compared to network from today (2021). It was one of the first CNNs that showed the potential of this structure for image recognition. Additionally...

Table of Contents

1. Summary
2. Network
3. Data Augmentation

Summary Çiçek et al. extended the idea of the u-net to 3D images. The kernels of the network were adjusted to the 3D by adding an additional dimension to the kernel size. The network learnes in to possibilities, once from user annotated slices or fully automated from a annotated training set. The images were of Xenopus kidneys. Application scenarios for volumetric segmentation with the 3D u-net. (a) Semi-automated segmentation: the user annotates some slices of each vol-ume to be segmented. The network predicts the dense segmentation. (b) Fully-automated segmentation: the network...