• Title/Summary/Keyword: Mixed Data Sampling

Search Result 108, Processing Time 0.023 seconds

A Generalized Mixed-Effects Model for Vaccination Data

  • Choi, Jae-Sung
    • Journal of the Korean Data and Information Science Society
    • /
    • v.15 no.2
    • /
    • pp.379-386
    • /
    • 2004
  • This paper deals with a mixed logit model for vaccination data. The effect of a newly developed vaccine for a certain chicken disease can be evaluated by a noninfection rate after injecting chicken with the disease vaccine. But there are a lot of factors that might affect the noninfecton rate. Some of these are fixed and others are random. Random factors are sometimes coming from the sampling scheme for choosing experimental units. This paper suggests a mixed model when some fixed factors need to have different experimental sizes by an experimental design and illustrates how to estimate parameters in a suggested model.

  • PDF

A Proportional Odds Mixed - Effects Model for Ordinal Data

  • Choi, Jae-Sung
    • Journal of the Korean Data and Information Science Society
    • /
    • v.18 no.2
    • /
    • pp.471-479
    • /
    • 2007
  • This paper discusses about how to build up mixed-effects model for analysing ordinal response data by using cumulative logits. Random factors are assumed to be coming from the designed sampling scheme for choosing observational units. Since the observed responses of individuals are ordinal, a proportional odds model with two random effects is suggested. Estimation procedure for the unknown parameters in a suggested model is also discussed by an illustrated example.

  • PDF

A study on intrusion detection performance improvement through imbalanced data processing (불균형 데이터 처리를 통한 침입탐지 성능향상에 관한 연구)

  • Jung, Il Ok;Ji, Jae-Won;Lee, Gyu-Hwan;Kim, Myo-Jeong
    • Convergence Security Journal
    • /
    • v.21 no.3
    • /
    • pp.57-66
    • /
    • 2021
  • As the detection performance using deep learning and machine learning of the intrusion detection field has been verified, the cases of using it are increasing day by day. However, it is difficult to collect the data required for learning, and it is difficult to apply the machine learning performance to reality due to the imbalance of the collected data. Therefore, in this paper, A mixed sampling technique using t-SNE visualization for imbalanced data processing is proposed as a solution to this problem. To do this, separate fields according to characteristics for intrusion detection events, including payload. Extracts TF-IDF-based features for separated fields. After applying the mixed sampling technique based on the extracted features, a data set optimized for intrusion detection with imbalanced data is obtained through data visualization using t-SNE. Nine sampling techniques were applied through the open intrusion detection dataset CSIC2012, and it was verified that the proposed sampling technique improves detection performance through F-score and G-mean evaluation indicators.

Anomaly Detection In Real Power Plant Vibration Data by MSCRED Base Model Improved By Subset Sampling Validation (Subset 샘플링 검증 기법을 활용한 MSCRED 모델 기반 발전소 진동 데이터의 이상 진단)

  • Hong, Su-Woong;Kwon, Jang-Woo
    • Journal of Convergence for Information Technology
    • /
    • v.12 no.1
    • /
    • pp.31-38
    • /
    • 2022
  • This paper applies an expert independent unsupervised neural network learning-based multivariate time series data analysis model, MSCRED(Multi-Scale Convolutional Recurrent Encoder-Decoder), and to overcome the limitation, because the MCRED is based on Auto-encoder model, that train data must not to be contaminated, by using learning data sampling technique, called Subset Sampling Validation. By using the vibration data of power plant equipment that has been labeled, the classification performance of MSCRED is evaluated with the Anomaly Score in many cases, 1) the abnormal data is mixed with the training data 2) when the abnormal data is removed from the training data in case 1. Through this, this paper presents an expert-independent anomaly diagnosis framework that is strong against error data, and presents a concise and accurate solution in various fields of multivariate time series data.

A mixed-effects model for overdispersed binomial data (초과변동의 이항자료에 대한 혼합효과 모형)

  • Choi, Jae-Sung
    • Journal of the Korean Data and Information Science Society
    • /
    • v.10 no.1
    • /
    • pp.199-205
    • /
    • 1999
  • This paper discusses the generalized mixed-effects model for the analysis of overdispersed binomial data. Sometimes certain types of sampling designs or genetic characters of experimental units can be regarded as factors of extra binomial variation. For such cases, this paper suggests models with one or two random effects to explain overdispersion caused by those affecting factors and shows how to test for a model adequacy based on deviance.

  • PDF

Supremacy of Realized Variance MIDAS Regression in Volatility Forecasting of Mutual Funds: Empirical Evidence From Malaysia

  • WAN, Cheong Kin;CHOO, Wei Chong;HO, Jen Sim;ZHANG, Yuruixian
    • The Journal of Asian Finance, Economics and Business
    • /
    • v.9 no.7
    • /
    • pp.1-15
    • /
    • 2022
  • Combining the strength of both Mixed Data Sampling (MIDAS) Regression and realized variance measures, this paper seeks to investigate two objectives: (1) evaluate the post-sample performance of the proposed weekly Realized Variance-MIDAS (RVar-MIDAS) in one-week ahead volatility forecasting against the established Generalized Autoregressive Conditional Heteroskedasticity (GARCH) model and the less explored but robust STES (Smooth Transition Exponential Smoothing) methods. (2) comparing forecast error performance between realized variance and squared residuals measures as a proxy for actual volatility. Data of seven private equity mutual fund indices (generated from 57 individual funds) from two different time periods (with and without financial crisis) are applied to 21 models. Robustness of the post-sample volatility forecasting of all models is validated by the Model Confidence Set (MCS) Procedures and revealed: (1) The weekly RVar-MIDAS model emerged as the best model, outperformed the robust DAILY-STES methods, and the weekly DAILY-GARCH models, particularly during a volatile period. (2) models with realized variance measured in estimation and as a proxy for actual volatility outperformed those using squared residual. This study contributes an empirical approach to one-week ahead volatility forecasting of mutual funds return, which is less explored in past literature on financial volatility forecasting compared to stocks volatility.

AN APPROACH TO THE TRAINING OF A SUPPORT VECTOR MACHINE (SVM) CLASSIFIER USING SMALL MIXED PIXELS

  • Yu, Byeong-Hyeok;Chi, Kwang-Hoon
    • Proceedings of the KSRS Conference
    • /
    • 2008.10a
    • /
    • pp.386-389
    • /
    • 2008
  • It is important that the training stage of a supervised classification is designed to provide the spectral information. On the design of the training stage of a classification typically calls for the use of a large sample of randomly selected pure pixels in order to characterize the classes. Such guidance is generally made without regard to the specific nature of the application in-hand, including the classifier to be used. An approach to the training of a support vector machine (SVM) classifier that is the opposite of that generally promoted for training set design is suggested. This approach uses a small sample of mixed spectral responses drawn from purposefully selected locations (geographical boundaries) in training. A sample of such data should, however, be easier and cheaper to acquire than that suggested by traditional approaches. In this research, we evaluated them against traditional approaches with high-resolution satellite data. The results proved that it can be used small mixed pixels to derive a classification with similar accuracy using a large number of pure pixels. The approach can also reduce substantial costs in training data acquisition because the sampling locations used are commonly easy to observe.

  • PDF

Methods and Techniques for Variance Component Estimation in Animal Breeding - Review -

  • Lee, C.
    • Asian-Australasian Journal of Animal Sciences
    • /
    • v.13 no.3
    • /
    • pp.413-422
    • /
    • 2000
  • In the class of models which include random effects, the variance component estimates are important to obtain accurate predictors and estimators. Variance component estimation is straightforward for balanced data but not for unbalanced data. Since orthogonality among factors is absent in unbalanced data, various methods for variance component estimation are available. REML estimation is the most widely used method in animal breeding because of its attractive statistical properties. Recently, Bayesian approach became feasible through Markov Chain Monte Carlo methods with increasingly powerful computers. Furthermore, advances in variance component estimation with complicated models such as generalized linear mixed models enabled animal breeders to analyze non-normal data.

Bayesian Approach for Software Reliability Models (소프트웨어 신뢰모형에 대한 베이지안 접근)

  • Choi, Ki-Heon
    • Journal of the Korean Data and Information Science Society
    • /
    • v.10 no.1
    • /
    • pp.119-133
    • /
    • 1999
  • A Markov Chain Monte Carlo method is developed to compute the software reliability model. We consider computation problem for determining of posterior distibution in Bayseian inference. Metropolis algorithms along with Gibbs sampling are proposed to preform the Bayesian inference of the Mixed model with record value statistics. For model determiniation, we explored the prequential conditional predictive ordinate criterion that selects the best model with the largest posterior likelihood among models using all possible subsets of the component intensity functions. To relax the monotonic intensity function assumptions. A numerical example with simulated data set is given.

  • PDF

Study on the Contribution of Mixing Effects in Sampling Tube and Condensation Nuclei Counter(CNC) to the measurement of size distribution obtained using Differential Mobility Analyzer and CNC (Differential Mobility Analyzer(DMA)와 Condensation Nuclei Counter(CNC)를 이용한 입자크기 분포 측정에서 샘플링 튜브와 CNC에서의 혼합 효과가 입자 크기 분포 측정에 미치는 영향에 관한 연구)

  • Lee, Youn-Soo;Ahn, Kang-Ho
    • Proceedings of the KSME Conference
    • /
    • 2001.06d
    • /
    • pp.104-109
    • /
    • 2001
  • The time to measure the size distribution using Condensation Nuclei Counter(CNC) and Differential Mobility Analyzer(DMA) can be shortened by classifying particles ramping the DMA voltage exponentially and continuously. In measurement, particles sampled at different time are mixed together going through sampling tube and CNC. Because the size distribution is inversed by using detector responses to sampling time intervals in this accelerated method, the mixing effects give inversion errors to the size distribution. The mixing effects can be considered by appling the transfer function with mixing effects to the data inversion. The inversion considering this effects gives birth to the size distribution shifted to the opposite direction of the size scanning.

  • PDF